Face Recognition: A Literature Review

Authors: A. S. Tolba, A.H. El-Baz, A.A. El-Harby

Abstract:

The task of face recognition has been actively researched in recent years. This paper provides an up-to-date review of major human face recognition research. We first present an overview of face recognition and its applications. Then, a literature review of the most recent face recognition techniques is presented. Description and limitations of face databases which are used to test the performance of these face recognition algorithms are given. A brief summary of the face recognition vendor test (FRVT) 2002, a large scale evaluation of automatic face recognition technology, and its conclusions are also given. Finally, we give a summary of the research results.

Keywords: Combined classifiers, face recognition, graph matching, neural networks.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1334652

References:

[1] Francis Galton, "Personal identification and description," In Nature,pp. 173-177, June 21, 1888.
[2] W. Zaho, "Robust image based 3D face recognition," Ph.D. Thesis, Maryland University, 1999.
[3] R. Chellappa, C.L. Wilson and C. Sirohey, "Humain and machine recognition of faces: A survey," Proc. IEEE, vol. 83, no. 5, pp. 705-740, may1995.
[4] T. Fromherz, P. Stucki, M. Bichsel, "A survey of face recognition," MML Technical Report, No 97.01, Dept. of Computer Science, University of Zurich, Zurich, 1997.
[5] T. Riklin-Raviv and A. Shashua, "The Quotient image: Class based recognition and synthesis under varying illumination conditions," In CVPR, P. II: pp. 566-571,1999.
[6] G.j. Edwards, T.f. Cootes and C.J. Taylor, "Face recognition using active appearance models," In ECCV, 1998.
[7] T. Sim, R. Sukthankar, M. Mullin and S. Baluja, "Memory-based face recognition for vistor identification," In AFGR, 2000.
[8] T. Sim and T. Kanade, "Combing models and exemplars for face recognition: An illuminating example," In Proceeding Of Workshop on Models Versus Exemplars in Computer Vision, CUPR 2001.
[9] L. Sirovitch and M. Kirby, "Low-dimensional procedure for the characterization of human faces," Journal of the Optical Society of America A, vol. 2, pp. 519-524, 1987.
[10] M. Turk and A. Pentland "Face recognition using eigenfaces," In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp.586-591, 1991.
[11] P. Belhumeur, P. Hespanha, and D. Kriegman, "Eigenfaces vs fisherfaces: Recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, 1997.
[12] M. Fleming and G. Cottrell, "Categorization of faces using unsupervised feature extraction," In Proc. IEEE IJCNN International Joint Conference on Neural Networks, pp. 65-70, 1990.
[13] B. Moghaddam, W. Wahid, and A. Pentland, "Beyond eigenfaces: Probabilistic matching for face recognition," In Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp. 30-35,1998.
[14] A. Lanitis, C. Taylor, and T. Cootes, "Automatic interpretation and coding of face images using flexible models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 743 -756, 1997.
[15] K. Jonsson, J. Matas, J. Kittler, and Y. Li, "Learning support vectors for face verification and recognition," In Proc. IEEE International Conference on Automatic Face and Gesture recognition, pp. 208-213, 2000.
[16] R. Brunelli and T. Poggio, "Face recognition: Features versus templates," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1042-1052, 1993.
[17] D. J. Beymer, "Face recognition under varying pose," A.I. Memo 1461, Center for Biological and Computational Learning, M.I.T., Cambridge, MA, 1993.
[18] L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg, "Face recognition by elastic bunch graph matching," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 775 -779, 1997.
[19] A. Nefian and M. Hayes, " An embedded hmm-based approach for face detection and recognition," In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 6, pp. 3553-3556,1999.
[20] U.S. Department of Defense, "Facial Recognition Vendor Test, 2000,"Available: http://www.dodcounterdrug.com/facialrecognition/FRVT2000/frvt2000.htm.
[21] W. Zhao and R. Chellappa, " Robust face recognition using symmetric shape-from-hading," Technical Report CARTR -919, 1999., Center for Automation Research, University of Maryland, College Park, MD,1999.
[22] L. Zheng, "A new model-based lighting normalization algorithm and its application in face recognition," Master-s thesis, National University of Singapore, 2000.
[23] G.J. Edwards, T.F. Cootes, and C.J. Taylor, "Face recognition using active appearance models," In ECCV, 1998.
[24] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, "From few to many: Generative models for recognition under variable pose and illumination," In AFGR, 2000.
[25] D.B. Graham and N.M., "Allinson face recognition from unfamiliar views: Subspace methods and pose dependency," In AFGR, 1998.
[26] L. Sirovich and M. Kirby, "Low-Dimensional procedure for the characterisation of human faces," J. Optical Soc. of Am., vol. 4, pp. 519-524, 1987.
[27] M. Kirby and L. Sirovich, "Application of the Karhunen- Loève procedure for the characterisation of human faces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, pp. 831-835, Dec.1990.
[28] M. Turk and A. Pentland, "Eigenfaces for recognition," J. Cognitive Neuroscience, vol. 3, pp. 71-86, 1991.
[29] M.A. Grudin, "A compact multi-level model for the recognition of facial images," Ph.D. thesis, Liverpool John Moores Univ., 1997.
[30] L. Zhao and Y.H. Yang, "Theoretical analysis of illumination in pcabased vision systems," Pattern Recognition, vol. 32, pp. 547-564,1999.
[31] A. Pentland, B. Moghaddam, and T. Starner, "View-Based and modular eigenspaces for face recognition," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 84-91, 1994.
[32] K. Chang, K.W. Bowyer and S. Sarkar, "Comarison and combination of ear and face images in appearance-based biometrics," IEEE Trans. On Pattern analysis and machine intelligence, vol. 25, no. 9,September 2003.
[33] L. Hong and A. Jain, "Integrating faces and fingerprints for personal identification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1295-1307, Dec. 1998. World Academy of Science, Engineering and Technology 19 2008
[34] P. Verlinde, G. Matre, and E. Mayoraz, "Decision fusion using a multilinear classifier," Proc. Int-l Conf. Multisource-Multisensor Information Fusion, vol. 1, pp. 47-53, July 1998.
[35] T.J. Stonham, "Practical face recognition and verification with WISARD," Aspects of Face Processing, pp. 426-441, 1984.
[36] K.K. Sung and T. Poggio, "Learning human face detection in cluttered scenes," Computer Analysis of Image and patterns, pp. 432-439, 1995.
[37] S. Lawrence, C.L. Giles, A.C. Tsoi, and A.D. Back, "Face recognition: A convolutional neural-network approach," IEEE Trans. Neural Networks, vol. 8, pp. 98-113, 1997.
[38] J. Weng, J.S. Huang, and N. Ahuja, "Learning recognition and segmentation of 3D objects from 2D images," Proc. IEEE Int'l Conf. Computer Vision, pp. 121-128, 1993.
[39] S.H. Lin, S.Y. Kung, and L.J. Lin, "Face recognition/detection by probabilistic decision-based neural network," IEEE Trans. Neural Networks, vol. 8, pp. 114-132, 1997.
[40] S.Y. Kung and J.S. Taur, "Decision-Based neural networks with signal/image classification applications," IEEE Trans. Neural Networks, vol. 6, pp. 170-181, 1995.
[41] M. Lades, J.C. Vorbruggen, J. Buhmann, J. Lange, C. Von Der Malsburg, R.P. Wurtz, and M. Konen, "Distortion Invariant object recognition in the dynamic link architecture," IEEE Trans. Computers, vol. 42, pp. 300-311, 1993.
[42] L. Wiskott and C. von der Malsburg, "Recognizing faces by dynamic link matching," Neuroimage, vol. 4, pp. 514-518, 1996.
[43] F. Samaria and F. Fallside, "Face identification and feature extraction using hidden markov models," Image Processing: Theory andm Application, G. Vernazza, ed., Elsevier, 1993.
[44] F. Samaria and A.C. Harter, "Parameterisation of a stochastic model for human face identification," Proc. Second IEEE Workshop Applications of Computer Vision, 1994.
[45] S. Tamura, H. Kawa, and H. Mitsumoto, "Male/Female identification from 8_6 very low resolution face images by neural network," Pattern Recognition, vol. 29, pp. 331-335, 1996.
[46] T. Kanade, "Picture processing by computer complex and recognition of human faces," technical report, Dept. Information Science, Kyoto Univ., 1973.
[47] A.J. Goldstein, L.D. Harmon, and A.B. Lesk, "Identification of human faces," Proc. IEEE, vol. 59, pp. 748, 1971.
[48] Y. Kaya and K. Kobayashi, "A basic study on human face recognition," Frontiers of Pattern Recognition, S. Watanabe, ed., pp. 265, 1972.
[49] R. Bruneli and T. Poggio, "Face recognition: features versus templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, pp. 1042-1052, 1993.
[50] I.J. Cox, J. Ghosn, and P.N. Yianios, "Feature-Based face recognition using mixture-distance," Computer Vision and Pattern Recognition, 1996.
[51] B.S. Manjunath, R. Chellappa, and C. von der Malsburg, "A Feature based approach to face recognition," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 373-378, 1992.
[52] R.J. Baron, "Mechanism of human facial recognition," Int'l J. Man Machine Studies, vol. 15, pp. 137-178, 1981.
[53] M. Bichsel, "Strategies of robust object recognition for identification of human faces," Ph.D. thesis, idgenossischen Technischen Hochschule, Zurich, 1991.
[54] T. Vetter and T. Poggio, "Linear object classes and image synthesis from a single example image," IEEE Trans. Pattern Analysis and Machin Intelligence, Vol. 19, no. 7, pp. 733-742, July 1997.
[55] D. Beymer and T. Poggio, "Face recognition from one model view," Proc. Fifth Int-l Conf. Computer Vision, 1995.
[56] T. Vetter and V. Blanz, "Estimating coloured 3D face models fromfingle images: An example based approach," Proc. Conf. Computer Vision (ECCV -98), vol. II, 1998.
[57] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, "From few to many: Illumination cone models for face recognition under variable lighting and pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, 2001.
[58] W. Zhao and R. Chellappa, "SFS based view synthesis for robust face recognition," Proc. Int-l Conf. Automatic Face and Gesture Recognition, pp. 285-292, 2000.
[59] V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model," IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 25, no. 9, September 2003.
[60] B. Takács, "Comparing face images using the modified hausdorff distance," Pattern Recognition, vol. 31, pp. 1873-1881, 1998.
[61] Y. Gao and K.H. Leung, "Face recognition using line edge map," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 6, June 2002.
[62] M.K.H. Leung and Y.H. Yang, "Dynamic two-strip algorithm in curve fitting," Pattern Recognition, vol. 23, pp. 69-79, 1990.
[63] Bern Univ. Face Database, ftp://iamftp.unibe.ch/pub/Images/FaceImages/, 2002.
[64] Purdue Univ. Face Database, http://rvl1.ecn.purdue.edu/~aleix/aleix_face_DB.html, 2002.
[65] V.N. Vapnik, "The nature of statistical learning theory," New York: Springverlag, 1995.
[66] C.J. Lin, "On the convergence of the decomposition method for support vector machines," IEEE Transactions on Neural Networks,2001.
[67] G. Guo, S.Z. Li, and K. Chan, "Face recognition by support vector machines," In proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp. 196-201, 2000.
[68] P.J. Phillips, "Support vector machines applied to face recognition," Processing system 11, 1999.
[69] B. Heisele, P. Ho, and T. Poggio, "Face recognition with support vector machines: Global versus component-based approach," in International Conference on Computer Vision (ICCV'01), 2001.
[70] J. Huang, V. Blanz, and B. Heisele, "Face recognition using Component-Based support vector machine Classification and Morphable models," LNCS 2388, pp. 334-341, 2002.
[71] K.Jonsson, J. Mates, J. Kittler and Y.P. Li, "Learning support vectors for face verification and recognition," Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, pp. 208-213, Los Alamitos, USA, March 2000.
[72] G.D. Guo, H.J. Zhang, S.Z. Li. "Pairwise face recognition". In Proceedings of 8th IEEE International Conference on Computer Vision. Vancouver, Canada. July 9-12, 2001.
[73] O. Deniz, M. Castrillon, M. Hernandez, "Face recognition using independent component analysis and support vector machines," Pattern Recognition Letters, vol. 24, pp. 2153-2157, 2003.
[74] Y. Li, S. Gong and H. Liddell. Support vector regression and classification based multi-view face detection and recognition. In Proc. IEEE International Conference on Face and Gesture Recognition, Grenoble, France, March 2000.
[75] K. I. Kim, K. Jung, and J. Kim, "Face recognition using support vector machines with local correlation kernels," International Journal of Pattern Recognition and Artificial Intelligence, vol. 16 no. 1, pp. 97- 111, 2002.
[76] K. Jonsson, J. Kittler, Y. P. Li, and J. Matas, "Support vector machines for face authentication," in T. Pridmore and D. Elliman, editors, British Machine Vision Conference, pp. 543-553, 1999.
[77] Huang J., X. Shao, and H. Wechsler, "Face pose discrimination using support vector machines," 14th International Conference on Pattern Recognition, (ICPR), Brisbane, Queensland, Aus, 1998.
[78] S. Pang, D. Kim, S.Y. Bang, "Membership authentication in the dynamic group by face classification using SVM ensemble," Pattern Recognition Letters vol. 24, pp. 215-225, 2003.
[79] A.S. Tolba, " A parameter-based combined classifier for invariant face recognition," Cybernetics and Systems, vol. 31, pp. 289-302, 2000.
[80] A.S. Tolba, and A.N. Abu-Rezq, "Combined classifiers for invariant face recognition," Pattern Anal. Appl. Vol. 3, no. 4, pp. 289-302, 2000.
[81] Ho-Man Tang, Michael Lyu, and Irwin King, "Face recognition committee machine," In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003), pp. 837- 840, April 6-10, 2003.
[82] Rui Huang, Vladimir Pavlovic, and Dimitris N. Metaxas, "A hybrid face recognition method using Markov random fields," ICPR (3) , pp. 157-160, 2004.
[83] A.S. Tolba, A.H. El-Baz, and A.A. El-Harby, "A robust boosted parameter- based combined classifier for pattern recognition," submitted for publication.
[84] John A. Black, M. Gargesha, K. Kahol, P. Kuchi, Sethuraman Panchanathan," A Framework for performance evaluation of face World Academy of Science, Engineering and Technology 19 2008 1141 recognition algorithms," in Proceedings of the International Conference on ITCOM, Internet Multimedia Systems II, 2002.
[85] The FacePix reference image set is in the public domain, Available: http://cubic.asu.edu/vccl/imagesets/facepix.
[86] P.J. Phillips, P. Grother, R.J. Michaels, D.M. Blackburn, E. Tabassi, and M. Bone, "Face recognition Vendor Test 2002: Evaluation Report," NISTIR 6965, NAT. Inst. Of Standards and Technology 2003.
[87] The At & T Database of Faces , Available: http://www.uk.research.att.com/facedatabase.html
[88] The Oulu Physics database, Available: http://www.ee.oulu.fi/research/imag/color/pbfd.html
[89] The XM2VTS database, Available: http://www.ee.surrey.ac.uk/Reseach/VSSP/xm2vtsdb/
[90] The Yale database, Available: http://cvc.yale.edu/
[91] The Yale B database, Available: http://cvc.yale.edu/projects/yalefacesB/yalefacesB.html
[92] The MIT face database, Available: ftp://whitehapel.media.mit.edu/pub/images/
[93] The CMU PIE database, Available: http://www.ri.cmu.edu/projects/project_418.html
[94] The UMIST database, Available: http://images.ee.umist.ac.uk/danny/database.html
[95] The University of Stirling online database , Available: http://pics.psych.stir.ac.uk/
[96] The FERET database, Available: http://www.it1.nist.gov/iad/humanid/feret/
[97] Kuwait University Face Database, Available: http://www.sc.kuniv.edu.kw/lessons/9503587/dina.htm