Study of Deep Learning-Based Model for Recognizing Human Activities in IoT Applications

Authors: Tarunima Chatterjee, Pinaki Pratim Acharjya

Abstract:

Advanced neural network-based human activity recognition (HAR) system integration with Internet of Things technology is progressing quickly. This technique, which has important implications in the fields of fitness, healthcare, and smart home environments, correctly detects and categorizes human actions from sensor data using sensors and deep learning algorithms. This work presents an approach that combines multi-head CNNs with an attention mechanism, producing a detection rate of 95.4%. Traditional HAR systems are generally imprecise and inefficient. Data collection, spectrogram image conversion, feature extraction, optimization, and classification are all steps in the procedure. With its deep learning foundation, this HAR system has enormous potential for real-time activity monitoring, especially in the healthcare industry, where it may enhance safety and offer insightful data on user behaviour.

Keywords: Deep learning, Human Activity Recognition, HAR, Internet of Things, IoT, Convolutional Neural Networks, CNNs, Long Short-Term Memory, LSTM, neural machine translation, NMT, Inertial Measurement Unit, IMU, Gated Recurrent Units, GRUs.

References:

[1] Yadav, S. K., Tiwari, K., Pandey, H. M., & Akbar, S. A. (2021). A review of multimodal human activity recognition with special emphasis on classification, applications, challenges and future directions. In Knowledge-Based Systems (Vol. 223, p. 106970). Elsevier BV. https://doi.org/10.1016/j.knosys.2021.106970.
[2] Qiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022). Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges. In Information Fusion (Vol. 80, pp. 241–265). Elsevier BV. https://doi.org/10.1016/j.inffus.2021.11.006.
[3] Atzori, L., Iera, A., & Morabito, G. (2010). The Internet of Things: A survey. In Computer Networks (Vol. 54, Issue 15, pp. 2787–2805). Elsevier BV. https://doi.org/10.1016/j.comnet.2010.05.010.
[4] Huynh, T., & Schiele, B. (2005, October). Analyzing features for activity recognition. In Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies (pp. 159-163).
[5] Yang, J., Nguyen, M. N., San, P. P., Li, X., & Krishnaswamy, S. (2015, July). Deep convolutional neural networks on multichannel time series for human activity recognition. In Ijcai (Vol. 15, pp. 3995-4001).
[6] Abobakr, A., Hossny, M., & Nahavandi, S. (2018). A Skeleton-Free Fall Detection System From Depth Images Using Random Decision Forest. In IEEE Systems Journal (Vol. 12, Issue 3, pp. 2994–3005). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/jsyst.2017.2780260.
[7] Adame, T., Bel, A., Carreras, A., Melià-Seguí, J., Oliver, M., & Pous, R. (2018). CUIDATS: An RFID–WSN hybrid monitoring system for smart health care environments. In Future Generation Computer Systems (Vol. 78, pp. 602–615). Elsevier BV.
[8] Bulling, A., Blanke, U., & Schiele, B. (2014). A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR), 46(3), 1-33.
[9] Khan D, Alonazi M, Abdelhaq M, Al Mudawi N, Algarni A, Jalal A, Liu H. Robust human locomotion and localization activity recognition over multisensory. Front Physiol. 2024 Feb 21; 15:1344887. doi: 10.3389/fphys.2024.1344887. PMID: 38449788; PMCID: PMC10915014.
[10] Bulling, A., Blanke, U., & Schiele, B. (2014). A tutorial on human activity recognition using body-worn inertial sensors. In ACM Computing Surveys (Vol. 46, Issue 3, pp. 1–33). Association for Computing Machinery (ACM). https://doi.org/10.1145/2499621.
[11] Zhang, L., Yu, J., Gao, Z., & Ni, Q. (2024). A multi-channel hybrid deep learning framework for multi-sensor fusion enabled human activity recognition. Alexandria Engineering Journal, 91, 472-485.
[12] Krishna, K. S., & Paneerselvam, S. (2021, October). An implementation of hybrid cnn-lstm model for human activity recognition. In International Conference on Advances in Electrical and Computer Technologies (pp. 813-825). Singapore: Springer Nature Singapore.
[13] LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., & Jackel, L. (1989). Handwritten digit recognition with a back-propagation network. Advances in neural information processing systems, 2.
[14] Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., ... & Bengio, Y. (2015, June). Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning (pp. 2048-2057). PMLR.
[15] Kianoush, S., Savazzi, S., Vicentini, F., Rampa, V., & Giussani, M. (2017). Device-Free RF Human Body Fall Detection and Localization in Industrial Workplaces. In IEEE Internet of Things Journal (Vol. 4, Issue 2, pp. 351–362). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/jiot.2016.2624800
[16] Koley, S., Acharjya, P. P., Mukharjee, R., Roy, S., Das., S., Deep Learning: An Approach to Encounter Pandemic Effect of Novel Corona Virus (COVID19), in Convergence of Deep Learning in Cyber-IoT Systems and Security, pp. 183-205, Print ISBN:9781119857211 |Online ISBN:9781119857686, 04 November 2022, DOI: https://doi.org/10.1002/9781119857686.ch9, Wiley Publications.
[17] Chattopadhyay, S., Acharjya, P. P., Koner, C., A Morphological Approach to Match Two Color Images Using Deep Learning, Proc. IEEE sponsored International Conference on Smart Systems and Inventive Technology, pp. 217-220, Tirunelveli, Tamilnadu, India, December 2018.DOI: 10.1109/ICSSIT.2018.8748396
[18] Chattopadhyay, S., Acharjya, P. P., Koner, C., A Deep Learning Approach to Implement Slab Based Image Steganography Algorithm of RGB Images, Proc. IEEE sponsored International Conference on Inventive Computation Technologies, pp. pp. 734-736, Coimbatore, Tamilnadu, India, November, 2018.DOI: 10.1109/ICICT43934.2018.9034299
[19] Santanu Koley and Pinaki Pratim Acharjya (2022), "Concepts & Techniques in Deep Learning Applications in the field of IoT Systems and Security", in Convergence of Deep Learning in Cyber-IoT Systems and Security, pp. 303-348, Print ISBN:9781119857211 |Online ISBN:9781119857686, 04 November 2022, DOI: https://doi.org/10.1002/9781119857686.ch15, Wiley Publications.
[20] Das, S., Acharjya, P. P., Integrating Machine Learning to Enhance Online Learning Student Performance, in book, Social Reflections of Human-Computer Interaction in Education, Management, and Economics, pp. 1-15, ISBN13: 9798369330333, EISBN13: 9798369330340, IGI Global: International Academic publisher, July 2024. DOI: 10.4018/979-8-3693-3033-3
[21] Das, S., Acharjya, P. P., Machine Learning Integration to Analyse the Students Behavior in an Online Digital Learning System, book, "Integrating Artificial Intelligence in Education: Enhancing Teaching Practices for Future Learning, pp-pages 217-242, ISBN13: 9798369339442, EISBN13: 9798369339459, IGI Global: International Academic publisher, 2024. DOI: 10.4018/979-8-3693-3944-2
[22] Das, S., Acharjya, P. P., Utilizing Machine Learning to Increase Online Learning Instructors' Efficiency, in book, Social Reflections of Human-Computer Interaction in Education, Management, and Economics, pp. 40-54, ISBN13: 9798369330333, EISBN13: 9798369330340, IGI Global: International Academic publisher, July 2024. DOI: 10.4018/979-8-3693-3033-3