Real-time 3D Feature Extraction without Explicit 3D Object Reconstruction

Authors: Kwangjin Hong, Chulhan Lee, Keechul Jung, Kyoungsu Oh

Abstract:

For the communication between human and computer in an interactive computing environment, the gesture recognition is studied vigorously. Therefore, a lot of studies have proposed efficient methods about the recognition algorithm using 2D camera captured images. However, there is a limitation to these methods, such as the extracted features cannot fully represent the object in real world. Although many studies used 3D features instead of 2D features for more accurate gesture recognition, the problem, such as the processing time to generate 3D objects, is still unsolved in related researches. Therefore we propose a method to extract the 3D features combined with the 3D object reconstruction. This method uses the modified GPU-based visual hull generation algorithm which disables unnecessary processes, such as the texture calculation to generate three kinds of 3D projection maps as the 3D feature: a nearest boundary, a farthest boundary, and a thickness of the object projected on the base-plane. In the section of experimental results, we present results of proposed method on eight human postures: T shape, both hands up, right hand up, left hand up, hands front, stand, sit and bend, and compare the computational time of the proposed method with that of the previous methods.

Keywords: Fast 3D Feature Extraction, Gesture Recognition, Computer Vision.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1331633

References:

[1] J. Loffler, "Content-based retrieval of 3d models in distributed web databases by visual shape information," in Proc. 4th International Conf. Information Visualization, 2000, pp. 82.
[2] C.M. Cyr, B.B. Kimia, "A similarity-based aspect-graph approach to 3d object recognition," International J. Computer Vision, vol. 57, 2004, pp. 5-22
[3] P. Min, J. Chen, T. Funkhouser, "A 2d sketch interface for a 3d model search engine," in Proc. the International Conf. Computer Graphics and Interactive Techniques, 2002, pp. 138.
[4] C. Chu, I. Cohen, "Posture and gesture recognition using 3d body shapes decomposition," in Proc. the IEEE Computer Society Conf. CVPR, vol. 3, 2005, pp. 69.
[5] D. Kyoung, Y. Lee, W. Baek, E. Han, J. Yang, K. Jung, "Efficient 3d voxel reconstruction using pre-computing method for gesture recognition," in Proc. Korea-Japan Joint Workshop, 2006, pp. 67-73.
[6] T. Funkhouser, P. Min, M. Kazhdan, J. Chen, A. Halderman, D. Dobkin, D. Jacobs, "A search engine for 3d models," ACM Trans. Graphics, vol. 22, 2003, pp. 83-105.
[7] M. Hilaga, Y. Shinagawa, T. Kohmura, T. Kunii, "Topology matching for fully automatic similarity estimation of 3d shapes." in Proc. the 28th annual Conf. Computer graphics and interactive techniques, 2001, pp. 203-212.
[8] H. Sundar, D. Silver, N. Gagvani, S. Dickinson, "Skeleton based shape matching and retrieval," in Proc. International Conf. Shape Modeling International, 2003, pp. 130-139.
[9] N.D. Cornea, D. Silver, P. Min, "Curve-skeleton properties, applications and algorithms," IEEE Trans. Visualization and Computer Graphics, vol. 13, 2007, pp. 530-548.
[10] N.D. Cornea, D. Silver, X. Yuan, R. Balasubramanian, "Computing hierarchical curve-skeletons of 3d objects," in Proc. the Visual Computer, vol. 21, 2005, pp. 945-955.
[11] A. Brennecke, T. Isenberg, "3d shape matching using skeleton graphs," in Proc. Simulation and Visualization, vol. 13, 2004, pp. 299-310
[12] A. Laurentini, "The visual hull concept for silhouette-based image understanding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, 1994, pp. 150-162.
[13] M. Li, M. Magnor, H. Seidel, "Hardware-accelerated visual hull reconstruction and rendering," in Proc. Graphics Interface, 2003, pp. 65-71.
[14] R. Szeliski, "Rapid octree construction from image sequences," in Proc. CVGIP: Image Underst., vol. 58, 1993, pp. 23-32.
[15] W. Matusik, C. Buehler, L. McMillan, "Polyhedral visual hulls for real-time rendering," in Proc. the 12th Eurographics Workshop. Rendering Technique, 2001, pp. 115-126.
[16] C. Lee, J. Cho, K. Oh, "Hardware-accelerated jaggy-free visual hulls with silhouette maps," in Proc. the ACM Sym. Virtual Reality Software and Technology, 2006, pp. 87-90.
[17] C. Everitt, A. Rege, C. Cebenoyan, "Hardware shadow mapping," Technical report, NVIDIA.