A Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition

Authors: Mahmoud Elmezain, Ayoub Al-Hamadi, Jörg Appenrodt, Bernd Michaelis

Abstract:

Gesture recognition is a challenging task for extracting meaningful gesture from continuous hand motion. In this paper, we propose an automatic system that recognizes isolated gesture, in addition meaningful gesture from continuous hand motion for Arabic numbers from 0 to 9 in real-time based on Hidden Markov Models (HMM). In order to handle isolated gesture, HMM using Ergodic, Left-Right (LR) and Left-Right Banded (LRB) topologies is applied over the discrete vector feature that is extracted from stereo color image sequences. These topologies are considered to different number of states ranging from 3 to 10. A new system is developed to recognize the meaningful gesture based on zero-codeword detection with static velocity motion for continuous gesture. Therefore, the LRB topology in conjunction with Baum-Welch (BW) algorithm for training and forward algorithm with Viterbi path for testing presents the best performance. Experimental results show that the proposed system can successfully recognize isolated and meaningful gesture and achieve average rate recognition 98.6% and 94.29% respectively.

Keywords: Computer Vision & Image Processing, Gesture Recognition, Pattern Recognition, Application

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1055307

References:

[1] X. Deyou, A Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG, In International Conference on Pattern Recognition (ICPR 06), pp. 519-522, 2006.
[2] D. B. Nguyen, S. Enokida, and E. Toshiaki, Real-Time Hand Tracking and Gesture Recognition System, IGVIP05 Conference, CICC, pp. 362-368, 2005.
[3] N. P. Vassilia, and G. M. Konstantinos, On Feature Extraction and Sign Recognition for Greek Sign Language, Proceedings of the 7th IASTED International Conference Artificial Intelligence and Soft Computer, pp. 93-98, 2003.
[4] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences, The Journal of W S C G-08, Vol. 16(1), pp. 65-72, 2008.
[5] E. Holden, R. Owens, and G. Roy, Hand Movement Classification Using Adaptive Fuzzy Expert System, Journal of expert Systems Research and Application, Vol. 9(4), pp. 465-480, 1996.
[6] L. Nianjun, C. L. Brian, J. K. Peter, and A. D. Richard, Model Structure Selection & Training Algorithms for a HMM Gesture Recognition System, In International Workshop in Frontiers of Handwriting Recognition, pp. 100-106, 2004.
[7] H. Lee, and J. Kim, An HMM-Based Threshold Model Approach for Gesture Recognition, IEEE Trans.on Pattern Analysis and Machine Intelligence, Vol. 21(10), pp. 961-973, 1999.
[8] Y. Ming-Hsuan, and A. Narendra, Gaussian Mixture Modeling of Human Skin Color and Its Applications in Image and Video Databases, In the SPIE/EI&T Storage and Retrieval for Image and Video Databases, pp. 458-466, 1999.
[9] S. Askar, Y. Kondratyuk, K. Elazouzi, P. Kauff, and O. Schreer, Vision- Based Skin-Colour Segmentation of Moving Hands for Real-Time Application, 1st European CVMP, pp. 79-85, 2004.
[10] S. L. Phung, A. Bouzerdoum, and D. Chai, A Novel Skin Color Model in Y CbCr Color Space and its Application to Human Face Detection, In IEEE International Conference on Image Processing (ICIP), pp. 289-292,2002.
[11] Y. Raja, S. J. Mckenna, and S. Gong, Colour Model Selection and Adaptation in Dynamic Scenes, In Proceedings European Conference on Computer Vision, pp. 460-474, 1998.
[12] A. R. Richard, and F. W. Homer, Mixture Densities, Maximum Likelihood and the EM Algorithm, SIAM Review, Vol. 26(2), pp. 195-239, 1984.
[13] R. Niese, A. Al-Hamadi, and B. Michaelis, A Stereo and Color-based Method for Face Pose Estimation and Facial Feature Extraction, The IEEE 18th International Conference on Pattern Recognition, Vol. 1, pp. 299- 302, 2006.
[14] R. Niese, A. Al-Hamadi, and B. Michaelis, A Novel Method for 3D Face Detection and Normalization, Journal of Multimedia, Vol. 2(5), pp. 1-12, 2007.
[15] R. R. Lawrence, A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proceeding of the IEEE, Vol. 77(2), pp. 257-286, 1989.
[16] S. Mitra, and T. Acharya, Gesture Recognition: A Survey, IEEE Transactions on Systems, MAN, and Cybernetics, pp. 311-324, 2007.
[17] H. Yoon, J. Soh, Y. J. Bae, and H. S. Yang, Hand Gesture Recognition using Combined Features of Location, Angle and Velocity, Pattern Recognition 34(7), pp. 1491-1501, 2001.
[18] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Gesture Recognition for Alphabets from Hand Motion Trajectory Using Hidden Markov Models, The IEEE International Symposium on Signal Processing and Information Technology, pp. 1209-1214, 2007.