Emotion Classification for Students with Autism in Mathematics E-learning using Physiological and Facial Expression Measures

Authors: Hui-Chuan Chu, Min-Ju Liao, Wei-Kai Cheng, William Wei-Jen Tsai, Yuh-Min Chen

Abstract:

Avoiding learning failures in mathematics e-learning environments caused by emotional problems in students with autism has become an important topic for combining of special education with information and communications technology. This study presents an adaptive emotional adjustment model in mathematics e-learning for students with autism, emphasizing the lack of emotional perception in mathematics e-learning systems. In addition, an emotion classification for students with autism was developed by inducing emotions in mathematical learning environments to record changes in the physiological signals and facial expressions of students. Using these methods, 58 emotional features were obtained. These features were then processed using one-way ANOVA and information gain (IG). After reducing the feature dimension, methods of support vector machines (SVM), k-nearest neighbors (KNN), and classification and regression trees (CART) were used to classify four emotional categories: baseline, happy, angry, and anxious. After testing and comparisons, in a situation without feature selection, the accuracy rate of the SVM classification can reach as high as 79.3-%. After using IG to reduce the feature dimension, with only 28 features remaining, SVM still has a classification accuracy of 78.2-%. The results of this research could enhance the effectiveness of eLearning in special education.

Keywords: Emotion classification, Physiological and facial Expression measures, Students with autism, Mathematics e-learning.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1072169

References:

[1] M. M. Mazzocco, G. Myers, "Complexities in identifying and defining mathematics learning disabilities in primary school-age years," Annals of Dyslexia, 2003, vol.53 pp. 218-53.
[2] S. Swezey, "Book reviews-autism and ICT: a guide for teachers and parents", Journal of Computers & Education, 2003, vol. 40, pp. 95-96.
[3] S. Weider, S. Greenspan, "Can children with autism master the core deficits and become empathic, creative and reflective? A ten to fifteen year follow-up of a subgroup of children with autism spectrum disorders (ASD) who received a comprehensive developmental, individual-difference, relationship-based (DIR) approach." Journal of Developmental and Learning Disorder, 2005, vol. 9, pp. 39-61.
[4] J. Lerner, "Learning Disabilities-Theories, Diagnosis, and Teaching Strategies"(9th ed.), 2003, Boston, MA: Houghton Mifflin.
[5] D. Carnine, "Curricular interventions for teaching higher order thinking to all students: Introduction to the special series". Journal of Learning Disabilities, 1991, vol. 24, pp. 261-269.
[6] S. Farrugia, J. Hudson, "Anxiety in adolescents with Asperger syndrome: negative thoughts, behavioral problems, and life interference," Journal of Focus on Autism and Other Developmental Disabilities, 2006, vol. 21, pp. 25-35.
[7] C. N. Moridis, A. A. Economides, "Towards computer-aided affective learning systems: A literature review," Journal of Educational Computing Research, 2008, vol. 39, pp. 313-337.
[8] A. Mehrabian, "Communication without words," Journal of Psychology today, 1968, vol.2, no.4, pp.53-56.
[9] El. Kaliouby, R.W. Picard and S. Baron-Cohen, " Affective Computing and Autism, ", Issue Progress in Convergence: Technologies for Human, Annals of the New York Academy of Sciences, 2007, vol. 1093.
[10] A. Haag, S. Goronzy, P. Schaich, and J. Williams. "Emotion recognition using bio-sensors: First steps towards an automatic system," Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), Affective Dialogue Systems,2004, vol. 3068, pp. 36-48.
[11] K. Gouizi, F. B. Reguig, and C. Maaoui, "Analysis physiological signals for emotion recognition," in Proceedings of the Seventh International Workshop on Systems, Signal Processing and their Applications, 2011, pp. 147-150.
[12] J. Pan, WJ. Tompkins, "A real-time QRS detection algorithm," IEEE Transactions on Biomedical Engineering, 1985, vol. 32, pp. 230-236.
[13] R. E. Kleiger, P. K. Stein, M. S. Bosner, and J. N. Rottman, "Time domain measurements of heart rate variability," Journal of Cardiol Clin, 1992, vol. 10, pp. 487-498.
[14] S. Akselrod, D. Gordon, F. A. Ubel, D. C. Shannon, A. C. Berger and R. J. Cohen, "Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control," Journal of Science, 1981, vol. 213, pp. 220-222.
[15] G. M. Hagg, A. Luttmann, M.Jager, "Methodologies for evaluating electromyographic field data in ergonomics," Journal of Electromyography and Kinesiology, 2000, vol. 10, pp. 301-312.
[16] M. A. Navakatikyan, C. J. Barrett, G. A. Head, J. H. Ricketts, S. C. Malpas, " A Real-time Algorithm for the Quantification of Blood Pressure Waveforms, "2002, vol. 49, pp. 662-670.
[17] K. H. Kim, S. W. Bang and S. R. Kim. "Emotion recognition system using short-term monitoring of physiological signals," In Medical and Biological Engineering and Computing, 2004, vol. 42, pp. 419-427.
[18] Face API technical specifications brochure. Available: http://www. seeingmachines.com/pdfs/brochures/faceAPI-Brochure.pdf.
[19] S. K. G. Murthy, "Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey," Journal of Data Mining and Knowledge Discovery, 1998, vol. 2, pp.345-389.
[20] C. Hsu, C. Chang, and C. Lin, "A Practical Guide to Support Vector Classification," Department of Computer Science, National Taiwan University, Taiwan, 2008.
[21] C. L. Huang, M. C. Chen, and C. J. Wang, "Credit scoring with a data mining approach based on support vector machines," Journal of Expert Systems with Applications, 2007, vol. 33, pp. 847-856.
[22] Thought Technology Limited. .
[23] S. Baron-Cohen, "Theory of mind and autism: A fifteen year review. In S. Baron Cohen, H. Tager-Flusberg, & D. J. Cohen (Eds.) Understanding other minds: Perspectives from developmental cognitive neuroscience" Oxford University Press: Oxford, 2000, pp. 3-20.
[24] C. Liu, K. Conn, N. Sarkar and W. Stone, "Physiology-Based Affect Recognition for Computer-assisted Intervention of Children with Autism Spectrum Disorde,"Journal of Human-Computer Studies, 2008, vol. 66, Issue 9, pp. 662-677.
[25] D. Delen, G. Walker and A. Kadam, "Predicting breast cancer survivability: a comparison of three data mining methods," Journal of Artificial Intelligence in Medicine, 2005 , vol. 34, pp. 113-127.