Analysis of Feature Space for a 2d/3d Vision based Emotion Recognition Method

Authors: Robert Niese, Ayoub Al-Hamadi, Bernd Michaelis

Abstract:

In modern human computer interaction systems (HCI), emotion recognition is becoming an imperative characteristic. The quest for effective and reliable emotion recognition in HCI has resulted in a need for better face detection, feature extraction and classification. In this paper we present results of feature space analysis after briefly explaining our fully automatic vision based emotion recognition method. We demonstrate the compactness of the feature space and show how the 2d/3d based method achieves superior features for the purpose of emotion classification. Also it is exposed that through feature normalization a widely person independent feature space is created. As a consequence, the classifier architecture has only a minor influence on the classification result. This is particularly elucidated with the help of confusion matrices. For this purpose advanced classification algorithms, such as Support Vector Machines and Artificial Neural Networks are employed, as well as the simple k- Nearest Neighbor classifier.

Keywords: Facial expression analysis, Feature extraction, Image processing, Pattern Recognition, Application.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1331425

References:

[1] P.E. Ekman, W.V. Friesen. Facial Action Coding System. Consulting Psychologists Press, Palo Alto, CA, 1978.
[2] M. Kunz, V. Mylius, K. Schepelmann, S. Lautenbacher: Impact of age on the facial expression of pain. J of Psychosom Res 2008; 64:311-318.
[3] C.A. Gilbert, C.M. Lilley, K.D. Craig, P.J. McGrath, C.A. Court, S.M. Bennett, C.J. Montgomery: Postoperative Pain Expression in Preschool Children: Validation of the Child Facial Coding System. Clin J Pain 1999; 15:192-200.
[4] G. Littlewort, M.S. Bartlett, I. Fasel, K. Lee: Faces of Pain: Automated Measurement of Spontaneous Facial Expressions of Genuine and Posed Pain. In ICMI -07: Proceedings of the 9th international conference on Multimodal interfaces, pp. 15-21. 2007.
[5] N. Esau, L. Kleinjohann, B. Kleinjohann: Integration of Emotional Reactions on Human Facial Expressions into the Robot Head MEXI. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE/RSJ IROS 2007). 2007.
[6] C. Hu, R. Feris, M. Turk, Real-time View-based Face Alignment using Active Wavelet Networks, Proc. IEEE, IEEE Int'l Workshop on Analysis and Modeling of Faces and Gestures, pp.215, 2003.
[7] D. Vukadinovic, M. Pantic: Fully automatic facial feature point detection using Gabor feature based boosted classifiers, Systems, Man and Cybernetics, 2005 IEEE International Conference on, vol 2, pp. 1692- 1698, 2005.
[8] N. Esau, E. Wetzel, L. Kleinjohann, B. Kleinjohann: Real-Time Facial Expression Recognition Using a Fuzzy Emotion Model. In Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2007). 2007.
[9] A.B. Ashraf, S. Lucey, J. Cohn, T. Chen, Z. Ambadar, K. Prkachin, P. Solomon, B.J. Eheobald: The Painful Face - Pain Expression Recognition Using Active Appearance Models: In ICMI. 2007.
[10] M. Monwar, S. Rezaej, K. Prkachin: Eigenimages Based Pain Expression Recognition. In IAENG International Journal of Applied Mathematics, 36:2. 2007.
[11] S. Brahnam, C.F. Chuang, F.Y. Shih, M.R. Slack: SVM Classification of Neonatal Facial Images of Pain. Fuzzy Logic and Applications, WILF 2005, Crema, Italy, 15-17, 2005, LNCS, vol. 3849, 2006b.
[12] G. Littlewort, M.S. Bartlett, I. Fasel, J. Susskind, J. Movellan, Dynamics of Facial Expression Extracted Automatically from Video. Image and Vision Computing, vol. 24, pp. 615-625, 2006.
[13] Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human Computing and Machine Understanding of Human Behavior: A Survey, in Artificial Intelligence for Human Computing, 2007.
[14] P. Hancock, C. Frowd, E. Brodie, C. Niven: Recognition of Pain Expressions. Progress in Neural Processing, vol. 16, pp. 339-348, 2005.
[15] S.Z. Li., A.K. Jain: Handbook of Face Recognition, ISBN: 0-387-40595- X, 2005.
[16] P. Viola, M. Jones: Rapid object detection using a boosted cascade of simple features, Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, 2001.
[17] B. Lucas, T. Kanade: An Iterative Image Registration Technique with an Application to Stereo Vision, Proc. of 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674-679, 1981.
[18] J. Albertz, W. Kreiling: Photogrammetric Guide, Herbert Wichmann Verlag GmbH, Karlsruhe, 1989.
[19] A. Al-Hamadi, R. Niese, B. Michaelis: A robust approach for contour extraction and tracking of moving objects in video sequences, Signal processing, pattern recognition, and applications (Greece June 30 - July 2, 2003) proceedings: Acta Press, pp. 336 - 341, 2003.
[20] V. Blanz, T. Vetter: A Morphable Model for the Synthesis of 3D Faces, SIGGRAPH, Conference Proceedings, 187-194, 1999.
[21] P. Albrecht, B. Michaelis: Stereo Photogrammetry with Improved Spatial Resolution, ICPR, pp. 845, 1998.
[22] R. Niese, A. Al-Hamadi, B. Michaelis: A Stereo and Color-based Method for Face Pose Estimation and Facial Feature Extraction. ICPR 2006 (IEEE), The 18th International Conference on Pattern Recognition, 2006, Hong Kong, Volume: 1, pp. 299-302, 2006.
[23] S. Rusinkiewicz, M. Levoy: Efficient variants of the ICP algorithm, Proc. of the 3rd Int. Conf. on 3D Digital Imaging & Modeling, pp. 145- 152, 2001.
[24] R. Calow, B. Michaelis: Markerless Analysis of Human Gait with a Multi-Camera-System. Proceedings of IASTED International Conference Biomedical Engineering (BioMED), pp. 270-275, 2005.
[25] S. Wachter: Verfolgung von Personen, Universität Karlsruhe, Dissertation, ISBN-13: 978-3980321266, 1997.
[26] J.L. Bentley: Multidimensional binary search trees used for associative searching, ACM, 18, pp. 509-517, 1975.
[27] N. Cristianini and J.S Taylor, "An Introduction to Support Vector Machines and other kernel based learning methods", ISBN: 0-521-78019-X, 2001.
[28] R. Herbrich: Learning kernel Classifiers: theory and algorithms, ISBN:0-262-08306-X, 2003.
[29] T.F. Wu, C.J. Lin: Probability Estimates for Multi-class Classification by Pair wise Coupling, Journal of Machine Learning Research 5, 975- 1005, 2004.
[30] Chang, C.-C., Lin., C.-J., LIBSVM: a library for support vector machines, Available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2009.