On-line Recognition of Isolated Gestures of Flight Deck Officers (FDO)

Authors: Deniz T. Sodiri, Venkat V S S Sastry

Abstract:

The paper presents an on-line recognition machine (RM) for continuous/isolated, dynamic and static gestures that arise in Flight Deck Officer (FDO) training. RM is based on generic pattern recognition framework. Gestures are represented as templates using summary statistics. The proposed recognition algorithm exploits temporal and spatial characteristics of gestures via dynamic programming and Markovian process. The algorithm predicts corresponding index of incremental input data in the templates in an on-line mode. Accumulated consistency in the sequence of prediction provides a similarity measurement (Score) between input data and the templates. The algorithm provides an intuitive mechanism for automatic detection of start/end frames of continuous gestures. In the present paper, we consider isolated gestures. The performance of RM is evaluated using four datasets - artificial (W TTest), hand motion (Yang) and FDO (tracker, vision-based ). RM achieves comparable results which are in agreement with other on-line and off-line algorithms such as hidden Markov model (HMM) and dynamic time warping (DTW). The proposed algorithm has the additional advantage of providing timely feedback for training purposes.

Keywords: On-line Recognition Algorithm, IsolatedDynamic/Static Gesture Recognition, On-line Markovian/DynamicProgramming, Training in Virtual Environments.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1085527

References:

[1] Christopher M. Bishop. Neural networks for pattern recognition. Oxford University Press, 1996.
[2] Herve Bourlardy and Samy Bengio. The Handbook of Brain Theory and Neural Networks, chapter Hidden Markov Models and other Finite State Automata for Sequence Processing. The MIT Press, second edition, 2002.
[3] E. Keogh C. A. Ratanamahatana. Everything you know about dynamic time warping is wrong. Third Workshop on Mining Temporal and Sequential Data, in conjunction with the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004.
[4] Andrea Corradini. Dynamic time warping for off-line recognition of a small gesture vocabulary. In Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real- Time Systems (RATFG-RTS-01), page 82. IEEE Computer Society, 2001.
[5] S. Sidney Fels and Geoffrey E. Hinton. Glove-Talk: A neural network interface between a data-glove and a speech synthesizer. IEEE Transactions on Neural Networks, 4(1):2-8, January 1993.
[6] Yang-Hee Nam Jane Koh. Full-body motion recognition using principle component based target reduction. In KIPS(Korean Information Processing Society) Proceedings, volume Vol. 11 , no.1, pages 873-876, Korea, May 2004.
[7] Yang-Hee Nam Jane Koh, Eun-Woo Lee. Full-body motion recognition using multi-phase target reduction method. HCI 2004(Korean), 2004.
[8] Yangsheng Xu Jie Yang. Hidden markov model for gesture recognition. Technical Report CMU-RI-TR-94-10, The Robotics Institute, Carneige Melon University, 1994.
[9] M. W. Kadous. Temporal Classification: Extending the Classification Paradigm to Multivariate Time Series. PhD thesis, The University of New South Wales, School of Computer Science and Engineering, 2002.
[10] C. Lee and Y. Xu. Online, interactive learning of gestures for human/ robot interfaces, 1996.
[11] H. Li and M. Greenspan. Continuous time-varying gesture segmentation by dynamic time warping of compound gesture models. 2005.
[12] Kouichi Murakami and Hitomi Taguchi. Gesture recognition using recurrent neural networks. In CHI -91: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 237-242, New York, NY, USA, 1991. ACM Press.
[13] Y. Nam and K. Wohn. Recognition of space-time handgestures using hidden markov model, 1996.
[14] Vladimir Pavlovic, Rajeev Sharma, and Thomas S. Huang. Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):677-695, 1997.
[15] Rabiner L. R. A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE, 77, (2):257-286, Feb 1989.
[16] Gerhard Rigoll, Andreas Kosmala, and Stefan Eickeler. High performance real-time gesture recognition using hidden markov models. In Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction, pages 69-80, London, UK, 1998. Springer-Verlag.
[17] D T Sodiri and V V S S Sastry. On the interpretation of gestures arising in flight deck officers training. In Proceedings of the Thirteenth Conference on Behaviour Representation in Modelling and Simulation, 2004.
[18] Thomas Hain-Phil Woodland Steve Young, Gunnar Evermann. The HTK Book, 3.2.1. Cambridge Research Laboratory Ltd, 2002.
[19] M Turk. Handbook of virtual environments: Design, implementation, and applications, chapter Gesture recognition, pages 223-238. Mahwah, NJ: Lawrence Erlbaum Associates, Inc., 2002.
[20] P. Vamplew and A. Adams. Recognition and anticipation of hand motions using a recurrent neural network, 1995.
[21] Simei G. Wysoski, Marcus V. Lamar, Susumu Kuroyanagi, and Akira Iwata. A rotation invariant approach on static-gesture recognition using boundary histograms and neural networks.
[22] Kiyoung Yang and Cyrus Shahabi. A pca-based similarity measure for multivariate time series. In Proceedings of the 2nd ACM international workshop on Multimedia databases, pages 65-74. ACM Press, 2004.