-
CiteScore
5.0
Impact Factor
IECE Transactions on Internet of Things, 2024, Volume 2, Issue 4: 74-82

Free Access | Research Article | 19 November 2024
1 Beaconhouse International College, Islamabad, Pakistan
* Corresponding author: Bilal Mushtaq, email: [email protected]
Received: 02 September 2024, Accepted: 04 November 2024, Published: 19 November 2024  

Abstract
This paper reviews advancements in assistive technology for deaf and hard of hearing individuals, highlighting the Internet of Things' (IoTs) pivotal role in enhancing their daily lives. Despite progress in sign language technologies, communication barriers persist. To address these gaps, we developed a video-based American Sign Language (ASL) identification system. Our approach utilizes MediaPipe for hand tracking, OpenCV for image normalization, and Gesture Control Convolutional Neural Network (CNN) for gesture localization. Implemented in Python, the system records video streams, filters hand regions, and recognizes ASL letter gestures with high accuracy. Leveraging computer vision and machine learning, our system enhances user experience, breaks communication barriers, promotes inclusivity, and supports accessible technologies. This innovative solution empowers deaf and hard of hearing individuals to fully participate in their communities, contributing to a more inclusive and accessible environment.

Graphical Abstract
IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges

Keywords
convolutional neural networks
sign language
artificial intelligence
machine learning
American sign language
image processing
internet of things
home automation

References

[1] Yang, L. I., Huang, J., Feng, T. I. A. N., Hong-An, W. A. N. G., & Guo-Zhong, D. A. I. (2019). Gesture interaction in virtual reality. Virtual Reality & Intelligent Hardware, 1(1), 84-112.

[2] Kheratkar, N., Bhavani, S., Jarali, A., Pathak, A., & Kumbhar, S. (2020, May). Gesture controlled home automation using CNN. In 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 620-626). IEEE.

[3] Tomar, D., Nauni, D., Zaidi, A. M., & Kaur, M. (2023, November). Gesture-Controlled Home Automation for the Differently Abled: Enhanced Accessibility and Independence. In 2023 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS) (pp. 1560-1564). IEEE.

[4] Okokpujie, K., Jacinth, D., James, G. A., Okokpujie, I. P., & Vincent, A. A. (2023). An IoT-based multimodal real-time home control system for the physically challenged: Design and implementation. Inf. Dyn. Appl, 2(2), 90-100.

[5] Kurian, B., Regi, J., John, D., Hari, P., & Mahesh, T. Y. (2023, May). Visual Gesture-Based Home Automation. In 2023 3rd International Conference on Advances in Computing, Communication, Embedded and Secure Systems (ACCESS) (pp. 286-290). IEEE.

[6] Yang, Y. (2016). Gesture controlled user interface for elderly people (Master’s thesis, Oslo and Akershus University College of Applied Sciences).

[7] Davis, J., & Shah, M. (1994). Recognizing hand gestures. In Computer Vision—ECCV’94: Third European Conference on Computer Vision Stockholm, Sweden, May 2–6, 1994 Proceedings, Volume I 3 (pp. 331-340). Springer Berlin Heidelberg.

[8] Wu, Y., & Huang, T. S. (2001). Hand modeling, analysis and recognition. IEEE Signal Processing Magazine, 18(3), 51-60.

[9] Park, G., Chandrasegar, V. K., & Koh, J. (2023). Accuracy enhancement of hand gesture recognition using CNN. IEEE Access, 11, 26496-26501.

[10] Sun, J. H., Ji, T. T., Zhang, S. B., Yang, J. K., & Ji, G. R. (2018, December). Research on the hand gesture recognition based on deep learning. In 2018 12th International symposium on antennas, propagation and EM theory (ISAPE) (pp. 1-4). IEEE.

[11] Licsár, A., & Szirányi, T. (2004, August). Dynamic training of hand gesture recognition system. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. (Vol. 4, pp. 971-974). IEEE.

[12] Hendricks, D. (2014). The history of smart homes. IoT Evolution World.

[13] Varriale, L., Briganti, P., & Mele, S. (2020). Disability and home automation: insights and challenges within organizational settings. In Exploring Digital Ecosystems: Organizational and Human Challenges (pp. 47-66). Springer International Publishing.

[14] Mushtaq, B., Rehman, M. A., Hussain, A., & Abbass, M. J. (2023, March). A highly selective dual bandpass filter using couple line resonator for modern wireless communication systems. In 2023 4th International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) (pp. 1-5). IEEE.

[15] Asadullah, M., & Raza, A. (2016, November). An overview of home automation systems. In 2016 2nd international conference on robotics and artificial intelligence (ICRAI) (pp. 27-31). IEEE.

[16] Mushtaq, B., Rehman, M. A., Khalid, S., & Alhaisoni, M. (2023). Design of Tri-Band Bandpass Filter Using Modified X-Shaped Structure for IoT-Based Wireless Applications. IEEE Embedded Systems Letters, 16(2), 194-197.

[17] Kim, Y., & Toomajian, B. (2016). Hand gesture recognition using micro-Doppler signatures with convolutional neural network. IEEE Access, 4, 7125-7130.

[18] Abdul Rehman, M., Khalid, S., Mushtaq, B., & Idrees, M. (2022). Design of a novel compact highly selective wideband bandstop RF filter using dual path lossy resonator for next generation applications. Plos one, 17(10), e0273514.

[19] Katre, S. R., & Rojatkar, D. V. (2017). Home automation: past, present and future. International research journal of engineering and technology, 4(10), 343-346.

[20] Ahmed, A., Botsinis, P., Won, S., Yang, L. L., & Hanzo, L. (2018). EXIT Chart Aided Convergence Analysis of Recursive Soft $ m $-Sequence Initial Acquisition in Nakagami-m Fading Channels. IEEE Transactions on Vehicular Technology, 67(5), 4655-4660.

[21] Ahmed, A. (2019). Iterative initial synchronization in wireless communications (Doctoral dissertation, University of Southampton).

[22] Mushtaq, B., & Khalid, S. (2023). Design of miniaturized single and dual-band bandpass filters using diamond-shaped coupled line resonator for next-generation wireless systems. International Journal of Microwave and Wireless Technologies, 15(3), 375-383.

[23] Mushtaq, B., Khalid, S., & Rehman, M. A. (2022). Design of a compact novel stub loaded pentaband bandpass filter for next generation wireless RF front ends. IEEE Access, 10, 109919-109924.

[24] Ahmed, A., Ahmed, Q. Z., Almogren, A., Haider, S. K., & Rehman, A. U. (2021). Hybrid precoding aided fast frequency-hopping for millimeter-wave Communication. IEEE Access, 9, 149596-149608.

[25] UN DESA. 2023. The Sustainable Development Goals Report 2023: Special Edition. New York, USA: UN DESA. © UN DESA.

[26] Khokher, A., Mushtaq, B., Rehman, M. A., & Abbas, M. J. (2024). RF Planning And Optimization Of 5G On The City Campus (MUST) of Mirpur, Pakistan. IECE Transactions on Sensing, Communication, and Control, 1(1), 52-59.

[27] Rehman, M. A., Mushtaq, B., Khalid, S., & Iqbal, J. (2023, August). Design and Analysis of Broadband Bandstop Filter using Dual Path Capacitive Coupled Resonator for 4G and 5G Applications. In 2023 20th International Bhurban Conference on Applied Sciences and Technology (IBCAST) (pp. 385-392). IEEE.

[28] Idrees, M., Mushtaq, B., Rehman, M. A., Khalid, S., & Iqbal, J. (2023, August). Design of a Planar Four-Port Microstrip Triplexer using Stub-Loaded Coupled Line Resonator for Advanced Wireless Applications. In 2023 20th International Bhurban Conference on Applied Sciences and Technology (IBCAST) (pp. 398-403). IEEE.

[29] Rehman, M. A., Mushtaq, B., Khalid, S., & Rehman, M. U. (2024). Design of a miniaturized multi resonance resonator based highly selective dual wideband bandpass filter. Microelectronics Journal, 153, 106411.

[30] Idrees, M., Khalid, S., Abdulrehman, M., Mushtaq, B., Najam, A. I., & Alhaisoni, M. (2023). Design of a stub-loaded coupled line diplexer for IoT-based applications. IEEE Embedded Systems Letters, 16(2), 186-189.

[31] Abdul Rehman, M., Khalid, S., Mushtaq, B., Uddin, M., Iqbal, J., Abdelhaq, M., & Alsaqour, R. (2023). A Novel Synthesis of Quasi-Chebyshev Ultra-Wideband Bandpass Filter Using N th Order Stub Loaded Coupled-Line Resonator. Micromachines, 14(10), 1874.

[32] Rehman, A., Rahman, M. A., Aziz, N., Mushtaq, B., Abbass, M. J., Khalid, S., & Jan, A. Z. (2022). Design of Highly Selective Dual Band Band Stop Filter using Dual-Path Step Impedance Resonator. Pakistan Journal of Engineering and Technology, 5(2), 146-151.


Cite This Article
APA Style
Fatima, B., Mushtaq, B., Iqbal, M. A., & Ahmed, A. (2024). IoT-based Smart Home Automation Using Gesture Control and Machine Learning for Individuals with Auditory Challenges. IECE Transactions on Internet of Things, 2(4), 74–82. https://doi.org/10.62762/TIOT.2024.723193

Article Metrics
Citations:

Crossref

0

Scopus

0

Web of Science

0
Article Access Statistics:
Views: 118
PDF Downloads: 12

Publisher's Note
IECE stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions
IECE or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
IECE Transactions on Internet of Things

IECE Transactions on Internet of Things

ISSN: 2996-9298 (Online)

Email: [email protected]

Portico

Portico

All published articles are preserved here permanently:
https://www.portico.org/publishers/iece/

Copyright © 2024 Institute of Emerging and Computer Engineers Inc.