Hand Gesture Recognition Based on Combined Features Extraction

Authors: Mahmoud Elmezain, Ayoub Al-Hamadi, Bernd Michaelis

Abstract:

Hand gesture is an active area of research in the vision community, mainly for the purpose of sign language recognition and Human Computer Interaction. In this paper, we propose a system to recognize alphabet characters (A-Z) and numbers (0-9) in real-time from stereo color image sequences using Hidden Markov Models (HMMs). Our system is based on three main stages; automatic segmentation and preprocessing of the hand regions, feature extraction and classification. In automatic segmentation and preprocessing stage, color and 3D depth map are used to detect hands where the hand trajectory will take place in further step using Mean-shift algorithm and Kalman filter. In the feature extraction stage, 3D combined features of location, orientation and velocity with respected to Cartesian systems are used. And then, k-means clustering is employed for HMMs codeword. The final stage so-called classification, Baum- Welch algorithm is used to do a full train for HMMs parameters. The gesture of alphabets and numbers is recognized using Left-Right Banded model in conjunction with Viterbi algorithm. Experimental results demonstrate that, our system can successfully recognize hand gestures with 98.33% recognition rate.

Keywords: Gesture Recognition, Computer Vision & Image Processing, Pattern Recognition.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1055805

References:

[1] X. Deyou, A Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG, International Conference ICPR, pp. 519-522, 2006.
[2] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences, The Journal of WSCG, Vol. 16, No. 1, pp. 65-72, 2008.
[3] M. Elmezain, A. Al-Hamadi, J. Appenrodt, and B. Michaelis, A Hidden Markov Model-Based Continuous Gesture Recognition System for Hand Motion Trajectory, International Conference on Pattern Recognition (ICPR) pp. 1-4, 2008.
[4] M. Elmezain, A. Al-Hamadi, and B. Michaelis, A Novel System for Automatic Hand Gesture Spotting and Recognition in Stereo Color Image Sequences, The Journal of WSCG, Vol. 17, No. 1, pp. 89-96, 2009.
[5] E. Holden, R. Owens, and G. Roy, Hand Movement Classification Using Adaptive Fuzzy Expert System, The Journal of Expert Systems, Vol. 9(4), pp. 465-480, 1996.
[6] N. Liu, and B. C. Lovell, MMX-accelerated Real-Time Hand Tracking System, In IVCNZ, pp. 381-385, 2001.
[7] T. Nobuhiko, S. Nobutaka, and S. Yoshiaki, Extraction of Hand Features for Recognition of Sign Language Words, In International Conference of VI, pp. 391-398, 2002.
[8] D. Comaniciu, V. Ramesh, and P. Meer, Kernel-Based Object Tracking, The IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 25, pp. 564-577, 2003.
[9] N. P. Vassilia and G. M. Konstantinos, On Feature Extraction and Sign Recognition for Greek Sign Language, International Conference on Artificial Intelligence and Soft Computer, pp. 93-98, 2003.
[10] Y. Ho-Sub, S. Jung, J. B. Young, and S. Y. Hyun, Hand Gesture Recognition using Combined Features of Location, Angle and Velocity, Journal of Pattern Recognition, Vol. 34(7), pp. 1491-1501, 2001.
[11] L. Nianjun, C. L. Brian, J. K. Peter, and A. D. Richard Model Structure Selection & Training Algorithms for a HMM Gesture Recognition System,International Workshop IWFHR, pp. 100-105, 2004.
[12] D. B. Nguyen, S. Enokida, and E. Toshiaki, Real-Time Hand Tracking and Gesture Recognition System, In GVIP Conf., pp. 362-368, 2005.
[13] D. Comaniciu, V. Ramesh, and P. Meer, Real-Time Tracking of Non- Rigid Objects Using Mean Shift, In Conference CVPR, pp. 1-8, 2000.
[14] G. Welch, and G. Bishop, An Introduction to the Kalman Filter, In Technical Report, University of North Carolina at Chapel Hill, pp. 95- 041, 1995.
[15] R. R. Lawrence, A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proceeding of the IEEE, Vol. 77(2), pp. 257-286, 1989.
[16] S. Mitra, and T. Acharya, Gesture Recognition: A Survey, IEEE Transactions on Systems, MAN, and Cybernetics, pp. 311-324, 2007.
[17] M. Elmezain, A. Al-Hamadi, S. S. Pathan, and B. Michaelis, Spatio- Temporal Feature Extraction-Based Hand Gesture Recognition for Isolated American Sign Language and Arabic Numbers. IEEE Symposium on ISPA, pp. 254-259, 2009.
[18] M. Elmezain, A. Al-Hamadi, G. Krell, S. El-Etriby, and B. Michaelis, Gesture Recognition for Alphabets from Hand Motion Trajectory Using Hidden Markov Models, The IEEE International Symposium on Signal Processing and Information Technology, pp. 1209-1214, 2007.
[19] T. Kanungo, D. M. Mount, N. Netanyahu, C. Piatko, R. Silverman, and A. Y. Wu, An Efficient k-means Clustering Algorithm: Analysis and Implementation, IEEE Transaction on PAMI, Vol. 24, pp. 881-892, 2002.
[20] C. Ding and X. He, K-means Clustering via Principal Component Analysis, International Conference on ML, pp. 225-232, 2004.
[21] R. Niese, A. Al-Hamadi, and B. Michaelis, A Novel Method for 3D Face Detection and Normalization, Journal of Multimedia, Vol. 2, pp. 1-12, 2007.
[22] S. Khalid, U. Ilyas, S. Sarfaraz, and A. Ajaz, ABhattacharyya Coefficient in Correlation of Gary-Scale Objects, The Journal of Multimedia, Vol. 1, pp. 56-61, 2006.