3D Human Reconstruction over Cloud Based Image Data via AI and Machine Learning

Authors: Kaushik Sathupadi, Sandesh Achar

Abstract:

Human action recognition (HAR) modeling is a critical task in machine learning. These systems require better techniques for recognizing body parts and selecting optimal features based on vision sensors to identify complex action patterns efficiently. Still, there is a considerable gap and challenges between images and videos, such as brightness, motion variation, and random clutters. This paper proposes a robust approach for classifying human actions over cloud-based image data. First, we apply pre-processing and detection, human and outer shape detection techniques. Next, we extract valuable information in terms of cues. We extract two distinct features: fuzzy local binary patterns and sequence representation. Then, we applied a greedy, randomized adaptive search procedure for data optimization and dimension reduction, and for classification, we used a random forest. We tested our model on two benchmark datasets, AAMAZ and the KTH Multi-view Football datasets. Our HAR framework significantly outperforms the other state-of-the-art approaches and achieves a better recognition rate of 91% and 89.6% over the AAMAZ and KTH Multi-view Football datasets, respectively.

Keywords: Computer vision, human motion analysis, random forest, machine learning.

References:

[1] Gutchess, D.; Trajkovics, M.; Cohen-Solal, E.; Lyons, D.; Jain, A.K. A background model initialization algorithm for video surveillance. In Proceedings of the Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vancouver, BC, Canada, 7–14 July 2001; pp. 20017–142001.
[2] Mustafa, Z., Nsour, H., & ud din Tahir, S. B. (2023). Hand gesture recognition via deep data optimization and 3D reconstruction. PeerJ Computer Science, 9, e1619.
[3] Z. Sun, Q. Ke, H. Rahmani, M. Bennamoun, G. Wang and J. Liu, "Human Action Recognition from Various Data Modalities: A Review," in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
[4] S. Tanberk, Z. H. Kilimci, D. B. Tükel, M. Uysal and S. Akyokuş, "A Hybrid Deep Model Using Deep Learning and Dense Optical Flow Approaches for Human Activity Recognition," in IEEE Access, vol. 8, pp. 19799-19809, 2020.
[5] M.-N. Chapel and T. Bouwmans, “Moving objects detection with a moving camera: A comprehensive review,” Comput. Sci. Rev., vol. 38, p. 100310, 2020.
[6] S. Angadi and S. Nandyal, “Human identification system based on spatial and temporal features in the video surveillance system,” Int. J. Ambient Comput. Intell., vol. 11, no. 3, pp. 1–21, 2020.
[7] Wieczorek, G.; Tahir, S.B.u.d.; Akhter, I.; Kurek, J. Vehicle Detection and Recognition Approach in Multi-Scale Traffic Monitoring System via Graph-Based Data Optimization. Sensors 2023, 23, 1731.
[8] Kumar, S., Shailu, A., Jain, A., & Moparthi, N. R. (2022). Enhanced method of object tracing using extended Kalman filter via binary search algorithm. Journal of Information Technology Management, 14(Special Issue: Security and Resource Management challenges for Internet of Things), 180-199.
[9] Bhargavi, D.; Coyotl, E.P.; Gholami, S. Knock, knock. Who’s there?--Identifying football player jersey numbers with synthetic data arXiv 2022, arXiv:2203.00734.
[10] Gholami, S.; Khashe, S. Alexa, Predict My Flight Delay. arXiv 2022, arXiv:2208.09921.
[11] Gaidon, A., Harchaoui, Z., & Schmid, C. (2013). Temporal localization of actions with actoms. IEEE transactions on pattern analysis and machine intelligence, 35(11), 2782-2795.
[12] Wang, L., Qiao, Y., & Tang, X. (2013). Latent hierarchical model of temporal structure for complex activity classification. IEEE Transactions on Image Processing, 23(2), 810-822.
[13] Zhang, W., Xu, L., Duan, P., Gong, W., Lu, Q., & Yang, S. (2015). A video cloud platform combing online and offline cloud computing technologies. Personal and Ubiquitous Computing, 19, 1099-1110.
[14] Lan, T., Zhu, Y., Zamir, A. R., & Savarese, S. (2015). Action recognition by hierarchical mid-level action elements. In Proceedings of the IEEE international conference on computer vision (pp. 4552-4560).
[15] Azhar, S. (2021). Automating Industrial Communication Standards selection by using a Knowledge-based systems (Master's thesis).
[16] Chou, K. P., Prasad, M., Wu, D., Sharma, N., Li, D. L., Lin, Y. F., ... & Lin, C. T. (2018). Robust feature-based automated multi-view human action recognition system. IEEE Access, 6, 15283-15296.
[17] Gupta, A., & Tiwari, R. (2015). Face detection using modified Viola jones algorithm. International Journal of Recent Research in Mathematics Computer Science and Information Technology, 1(2), 59-66.
[18] Resende, M. G., & Ribeiro, C. C. (2010). Greedy randomized adaptive search procedures: Advances, hybridizations, and applications. Handbook of metaheuristics, 283-319.
[19] Itti, L., & Baldi, P. (2005, June). A principled approach to detecting surprising events in video. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) (Vol. 1, pp. 631-637). IEEE.
[20] Itti, L., & Baldi, P. (2005, June). A principled approach to detecting surprising events in video. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) (Vol. 1, pp. 631-637). IEEE.
[21] Nadeem, A., Jalal, A., & Kim, K. (2021). Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy markov model. Multimedia Tools and Applications, 80, 21465-21498.
[22] Jiang, Y.; Tong, G.; Yin, H.; Xiong, N. A Pedestrian Detection Method Based on Genetic Algorithm for Optimize XGBoost Training Parameters. IEEE Access 2019, 7, 118310–118321.
[23] Subasi, A.; Dammas, DH; Alghamdi, RD; Makawi, R.A.; Albiety, E.A.; Brahimi, T.; Sarirete, A. Sensor Based Human Activity Recognition Using AdaBoost Ensemble Classifier. Procedia Comput. Sci. 2018, 140, 104–111.
[24] M. H. Oreaba, "Solving the confusion of body sides problem in two-dimensional human pose estimation", Master's Thesis, the American University, 2017.
[25] V. Belagiannis, S. Amin, M. Andriluka, B. Schiele, N. Navab and S. Ilic, "3D pictorial structures for multiple human pose estimation", Proc. CVPR, June 2014.
[26] H. W. Chen and M. MeGurr. "Moving human full body and body parts detection, tracking, and applications on human activity estimation." SPIE Defense + Security, 2016.
[27] Xia, L.; Chen, C.; Aggarwal, J.K. View invariant human action recognition using histograms of 3D joints. In Proceedings of the Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 20–27.
[28] Han, Y.; Chung, S.L.; Ambikapathi, A.; Chan, J.S.; Lin, W.Y.; Su, S.F. Robust human action recognition using global spatial-temporal attention for human skeleton data. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018.
[29] Das, S.; Chaudhary, A.; Bremond, F.; Thonnat, M. Where to focus on for human action recognition? In Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa Village, HI, USA, 7–11 January 2019; pp. 71–80.