Deep-Learning Based Approach to Facial Emotion Recognition Through Convolutional Neural Network

Authors: Nouha Khediri, Mohammed Ben Ammar, Monji Kherallah

Abstract:

Recently, facial emotion recognition (FER) has become increasingly essential to understand the state of the human mind. However, accurately classifying emotion from the face is a challenging task. In this paper, we present a facial emotion recognition approach named CV-FER benefiting from deep learning, especially CNN and VGG16. First, the data are pre-processed with data cleaning and data rotation. Then, we augment the data and proceed to our FER model, which contains five convolutions layers and five pooling layers. Finally, a softmax classifier is used in the output layer to recognize emotions. Based on the above contents, this paper reviews the works of facial emotion recognition based on deep learning. Experiments show that our model outperforms the other methods using the same FER2013 database and yields a recognition rate of 92%. We also put forward some suggestions for future work.

Keywords: CNN, deep-learning, facial emotion recognition, machine learning.

References:

[1] N. Kh´ediri, M. Ben Ammar and M. Kherallah, ”Towards an online Emotional Recognition System for Intelligent Tutoring Environment.”, (ACIT’2017) The International Arab Conference on Information Technology, Yassmine Hammamet, Tunisia, 22-24 December 2017.
[2] M. Moolchandani, S. Dwivedi, S. Nigam and K. Gupta, ”A survey on: Facial Emotion Recognition and Classification,” 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), 2021, pp. 1677-1686.
[3] Naga, P., Marri, S.D. and Borreo, R., 2021. ”Facial emotion recognition methods, datasets and technologies: A literature survey”. Materials Today: Proceedings.
[4] Nair, V., Hinton, G.E.: ”Rectified linear units improve restricted boltzmann machines”. In: ICML. pp. 807–814 (2010).
[5] Singh M., Sharma S., Paul S., Sajeevan J., and Paul S., ”Facial Emotion Recognition system” Journal of Scientific Research and Advances, volume 6, issue 6,2020.
[6] Mahmoudi, M., Chetouani, A., Boufera, F., and Tabia, H. (2020), ”Improved Bilinear Model for Facial Expression Recognition”, Pattern Recognition and Artificial Intelligence, 1322, 47 - 59.
[7] Hao Meng, Fei Yuan, Yue Wu and Tianhao Yan, ”Facial Expression Recognition Algorithm Based on Fusion of Transformed Multilevel Features and Improved Weighted Voting SVM”, Mathematical Problems in Engineering,1-117,2021.
[8] Ben Niu, Zhenxing Gao and Bingbing Guo, ”Facial Expression Recognition with LBP and ORB Features”, Computational Intelligence and Neuroscience,2021.
[9] Saroop, A., Ghugare, P., Mathamsetty, S., and Vasani, V. (2021), ”Facial Emotion Recognition: A multi-task approach using deep learning”. ArXiv, abs/2110.15028.
[10] Minaee, S., and Abdolrashidi, A. (2021). Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network. Sensors (Basel, Switzerland), 21.
[11] Zeiler, D.M.; Fergus, R., ”Visualizing and understanding convolutional networks”, In European Conference on Computer Vision;Springer: Cham, Switzerland, 2014.
[12] I. J. Goodfellow, D. Erhan, P. L. Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y. Tang, D. Thaler, D.-H. Lee et al., “Challenges in representation learning: A report on three machine learning contests,” in International Conference on Neural Information Processing. Springer, 2013, pp. 117–124.
[13] Simonyan K. and Zisserman A., ”Very Deep Convolutional Networks for Large-Scale Image Recognition”, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
[14] Diederik P. Kingma, Jimmy Ba, ”Adam: A Method for Stochastic Optimization”, 3rd International Conference for Learning Representations, San Diego, 2015.