Combining Skin Color and Optical Flow for Computer Vision Systems

Authors: Muhammad Raza Ali, Tim Morris

Abstract:

Skin color is an important visual cue for computer vision systems involving human users. In this paper we combine skin color and optical flow for detection and tracking of skin regions. We apply these techniques to gesture recognition with encouraging results. We propose a novel skin similarity measure. For grouping detected skin regions we propose a novel skin region grouping mechanism. The proposed techniques work with any number of skin regions making them suitable for a multiuser scenario.

Keywords: Bayesian tracking, chromaticity space, optical flowgesture recognition

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1332712

References:

[1] M.R. Ali and T. Morris. "Skin Locus Based Skin Detection for Gesture Recognition". BMVC Postgraduate Workshop, Aberystwyth, 3- Sep-2010.
[2] A. Argyros and M. Lourakis. "Real time Tracking of Multiple Skin- Colored Objects with a Possibly Moving Camera". In Proceedings of the European Conference on Computer Vision (ECCV-04), Springer- Verlag, vol. 3, pp. 368-379, Prague, Czech Republic, May 11-14, 2004.
[3] A. Del. Bimbo, L. Landucci , and A. Valli. "Multi-User Natural Interaction System based on Real-Time Hand Tracking and Gesture Recognition". In Proceedings of ICPR (3). 2006, pp: 55-58.
[4] M. Black and A.D. Jepson. "A probabilistic framework for matching temporal trajectories: Condensation-based recognition of gestures and expressions". In Proceedings of Fifth European Conference on Computer Vision Vol. 1, 1998, pp: 909-924.
[5] T. Coogan, G. Awad, J. Han, and A. Sutherland. "Real time hand gesture recognition including hand segmentation and tracking". In Proceedings of ISVC06, 2006 pp: 495 - 504.
[6] P. Dreuw, T. Deselaers, D. Keysers, D, and H. Ney. "Modelling image variability in appearance-based gesture recognition". In ECCV Workshop on Statistical Methods in Multi-Image and Video Processing (ECCV-SMVP), Graz, Austria, May 2006, pp: 7-18.
[7] A. Gelb. "Applied Optimal Estimation". MIT Press, 1996
[8] M.S. Grewal and A.P. Andrews "Kalman Filtering". Prentice Hall, 1993.
[9] C. Hsu, C. Chang, and C.J Lin. "A Practical Guide to Support Vector Classification". http://www.csie.ntu.edu.tw/~cjlin/
[10] M. Isard and A. Blake. "CONDENSATION -- Conditional Density Propagation For Visual Tracking". Int. Journal of Computer Vision, Vol: 29(1), 1998, pp: 5-28.
[11] E. Koller-Meier and F. Ade, "Tracking Multiple Objects Using the Condensation algorithm". Robotics and Autonomous Systems, vol. 34, 2001, pp. 93-105.
[12] B. Lucas and T. Kanade. "An Iterative Image Registration Technique With An Application To Stereo Vision". Proceedings of Imaging understanding workshop, 1981, pp: 121-130.
[13] L. Saigol , S. Sclaroff , and V. Athitsos. "Skin Color-Based Video Segmentation Under Time-Varying Illumination". IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 26, No. 7, July 2004.
[14] J. Santner , M. Werlberger, T. Mauthner, W. Paier , and Bischof H. "Flow Games". 1st Int. Workshop on Computer Vision for Computer Games (CVCG) in conjunction with IEEE CVPR, 2010.
[15] G. Shin. and J. Chun, "Vision-based Multimodal Human Computer Interface based on Parallel Tracking of Eye and Hand Motion". In Proceedings of International Conference on Convergence Information Technology , 2007, pp. 2443-2448.
[16] M. Soriano, B. Martinkauppi, S. Huovinen, and M. Laaksonen, "Adaptive Skin Color Modelling Using The Skin Locus For Selecting Training Pixels". Pattern Recognition,36(3), 2003, pp: 681-690.
[17] M. Storring , J.H. Andersen, and E. Granum, "Skin Color Detection Under Changing Lighting Conditions". In Proceedings of Seventh Symposium Intelligent Robotics Systems,1999. pp. 187-195.
[18] D. Sun, S. Roth and M. Black, Secrets of Optical Flow Estimation and Their Principles". In proceedings of CVPR, 2010. http://www.cs.brown.edu/~dqsun/pubs/cvpr_2010_flow.pdf
[19] N. Vo, Q. Tran, T.B. Dinh and Q.M. Nguyen, "An Efficient Human Computer Interaction Framework Using Skin Color Tracking and Gesture Recognition". International Conference on Computing and Communication Technologies, Research, Innovation, and Vision for the Future (RIVF), 2010.
[20] Y. Wu and T. Huang. "Vision Based Gesture Recognition: A review", In International Gesture Workshop (GW99),Gif-sur-Yvette, France, 1999.
[21] Y. Yin and D.R. Toward, "Natural Interaction In The Real World: Real-Time Gesture Recognition". International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI '10), Beijing 2010.
[22] S. Zhan, Y. Hanxuan, Z. Yanguo, and Z. Feng, "Hand detection and gesture recognition exploit motion times image in complicate scenarios". In Proceedings of the 6th international conference on Advances in visual computing - Volum. Part II. Springer-Verlag, Berlin, Heidelberg, 2010, 628-636.