3D Face Modeling based on 3D Dense Morphable Face Shape Model

Authors: Yongsuk Jang Kim, Sun-Tae Chung, Boogyun Kim, Seongwon Cho

Abstract:

Realistic 3D face model is more precise in representing pose, illumination, and expression of face than 2D face model so that it can be utilized usefully in various applications such as face recognition, games, avatars, animations, and etc. In this paper, we propose a 3D face modeling method based on 3D dense morphable shape model. The proposed 3D modeling method first constructs a 3D dense morphable shape model from 3D face scan data obtained using a 3D scanner. Next, the proposed method extracts and matches facial landmarks from 2D image sequence containing a face to be modeled, and then reconstructs 3D vertices coordinates of the landmarks using a factorization-based SfM technique. Then, the proposed method obtains a 3D dense shape model of the face to be modeled by fitting the constructed 3D dense morphable shape model into the reconstructed 3D vertices. Also, the proposed method makes a cylindrical texture map using 2D face image sequence. Finally, the proposed method generates a 3D face model by rendering the 3D dense face shape model using the cylindrical texture map. Through building processes of 3D face model by the proposed method, it is shown that the proposed method is relatively easy, fast and precise.

Keywords: 3D Face Modeling, 3D Morphable Shape Model, 3DReconstruction, 3D Correspondence.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1054923

References:

[1] W. Zhao and R. Chellappa, Face Processing: Advanced Modeling and Methods, Elsevier, 2005.
[2] Y. Lee, D. Terzopuolos, K. Waters, "Realistic Modeling for Facial Animation", Proc. SIGGRAPH, Los Angeles, pp.55-61, August, 1995.
[3] F. I. Parke and K. Waters, "Appendix 1: Three-dimensional muscle model facial animation", Computer Facial Animation, A.K. Peters, Sept. 1996.
[4] F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin, "Synthesizing realistic facial expressions from photographs", In Computer Graphics, Annual Conference Series, SIGGRAPH, pp75-84, July, 1998.
[5] V. Blanz, T. Vetter, "A Morphable Model for the Synthesis of 3D Faces", Proc. of the SIGGRAPH'99, August 1999, Los Angeles, USA, pp.187-194, 1999.
[6] J. Ahlberg, "CANDIDE-3 -- an updated parameterized face", Report No. LiTH-ISY -R-2326, Dept. of Electrical Engineering, Linköping University, Sweden, 2001.
[7] R. L. Hsu, A. K. Jain, "Face Modeling for Recognition", Proc. Int'l Conf. Image Processing (ICIP), Vol.2, pp.693-696, 2001.
[8] A. Ansari and M. Abdel-Mottaleb, "3-D Face Modeling Using Two Views and a Generic Face Model with Application to 3-D Face Recognition", IEEE Conf. on Advanced Video and Signal Based Surveillance, pp.203-222, 2003.
[9] Y. Hu, D. Jiang, S. Yan, L. Zhang, H. zhang, "Automatic 3D reconstruction for face recognition", Proc. 6th IEEE Int'l Conf. on Automatic Face and Gesture Recognition, pp.843-848, 2004.
[10] Mandun Zhang Linna Ma Xiangyong Zeng Yangsheng, "Image-Based 3D Face Modeling ", Proc. of Int'l Conf. on Computer Graphics, Imaging and Visualization, pp.165-168, July 2004.
[11] H. Guo, J. Jiang and L. Zhang, "Building a 3D morphable face model by using thin plate splines for face reconstruction", LNCS Vol. 3338, pp.258-267, 2004.
[12] Z. Zhang, Z. Liu, D. Adler, M. F. Cohen, E. Hanson, and Y. Shan, "Robust and Rapid Generation of Animated Faces from Video Images: A Model-Based Modeling Approach", International Journal of Computer Vision, Vol.58, No.2, pp.93-119, June 2004.
[13] T. Russ, C. Boehnen, T. Peters, "3D Face Recognition Using 3D Alignment for PCA", IEEE Conf. on Computer Vision and Pattern Recognition, Vol.2, pp.1391-1398, 2006.
[14] Cyberware, http://www.cyberware.com/
[15] M. B. Stegmann, Mikkel B., Gomez , David Delgado: A Brief Introduction to Statistical Shape Analysis Technical University of Denmark, Lyngby, 2002.
[16] T. F. Cootes, D. J. Edwards, S. J. Taylor, "Active Appearance Models", IEEE Trans. Pattern Anal. Mach. Intell, Vol. 23, No. 6, pp. 681-685, Jun. 2001.
[17] Besl, P. and McKay, N. "A Method for Registration of 3-D Shapes", Trans. PAMI, Vol. 14, No. 2, 1992.
[18] S. Rusinkewicz and M. Levoy, "Efficient Variants of the ICP Algorithm", Third International Conference on 3D Digital Imaging and Modeling, pp.145-152, June, 2001.
[19] B. Brown and S. Rusinkiewicz. "Non-Rigid Range-Scan Alignment Using Thin-Plate Splines", Symposium on 3D Data Processing. Visualization and Transmission, Vol.6, No.9, pp.759-765, September 2004.
[20] F. L. Bookstein., "Principal warps: Thin-plate splines and the decomposition of deforma tions", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.11, No.6, 567-585, June 1989.
[21] R. Hartley and A. Zisserman, Multiple Geometry in Compution Vision, 2nd ed. Cambridge University Press, 2003.
[22] C. Tomasi and T. Kanade, "Shape and motion from image streams under orthography: A factorization method", International Journal of Computer Vision, Vol.9, No.2, pp.137-154, 1992.
[23] Conrad J. Poelman and Takeo Kanade, "A Paraperspective Factorization Method for Shape and Motion Recovery", IEEE PAMI, Vol.19, No.3, March, 1997.
[24] S. Mahamud and M. Hebert, "Iterative projective reconstruction from multiple views", Proc. IEEE Conf. Computer Vision and Pattern Recognition, Vol.2, pp.430-437, 2000.
[25] T. Jebara, A. Azarbayejani, and A. Pentland, "3D structure from 2D motion", IEEE Signal Processing Magazine, Vol.16, Issue.3, pp.66-84, May 1999.
[26] C. Brechbuhler, G. Gerig, G., O. Kubler, O.: "Parameterization of Closed Surfaces for 3-D Shape Description", Comp. Vision and Image Understanding, Vol.61, Issue.2, pp.154-170. 1995.
[27] R. Davies, C. Twining, T. Cootes, J. Waterton, and C. Taylor. "A minimum description length approach to statistical shape modeling", IEEE Transactions on Medical Imaging, Vol.5, Issue.5, pp.525-537, May 2002.