Predicting the Three Major Dimensions of the Learner-s Emotions from Brainwaves

Authors: Alicia Heraz, Claude Frasson

Abstract:

This paper investigates how the use of machine learning techniques can significantly predict the three major dimensions of learner-s emotions (pleasure, arousal and dominance) from brainwaves. This study has adopted an experimentation in which participants were exposed to a set of pictures from the International Affective Picture System (IAPS) while their electrical brain activity was recorded with an electroencephalogram (EEG). The pictures were already rated in a previous study via the affective rating system Self-Assessment Manikin (SAM) to assess the three dimensions of pleasure, arousal, and dominance. For each picture, we took the mean of these values for all subjects used in this previous study and associated them to the recorded brainwaves of the participants in our study. Correlation and regression analyses confirmed the hypothesis that brainwave measures could significantly predict emotional dimensions. This can be very useful in the case of impassive, taciturn or disabled learners. Standard classification techniques were used to assess the reliability of the automatic detection of learners- three major dimensions from the brainwaves. We discuss the results and the pertinence of such a method to assess learner-s emotions and integrate it into a brainwavesensing Intelligent Tutoring System.

Keywords: Algorithms, brainwaves, emotional dimensions, performance.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1076762

References:

[1] Aha, D., & Kibler, D. Instance-based learning algorithms. Machine Learning, 6, 37-66, 1991
[2] Anderson T. W. An Introduction to Multivariate Statistical Analysis, 3rd ed. Wiley, New York, 2003.
[3] Bear, M. F., Connors, B. W., & Paradiso, M. A. Neuroscience: Exploring the Brain, second ed. Lippincott Williams & Williams, Baltimore, MD, 2001.
[4] Bellifemine, F., A. Poggi, & G. Rimassa, "JADE - A FIPA-compliant Agent Framework", PAAM '99, London, UK, 1999, pp. 97ÔÇö108, 1999.
[5] Breazeal, C. Designing sociable robots .Cambridge: MIT Press, 2003.
[6] Breiman L. Bagging predictors. Machine Learning, 24(2):123-140, 1996.
[7] Cantor, D. S. An overview of quantitative EEG and its applications to neurofeedback. In Introduction to Quantitative EEG and Neurofeedback, J. R. Evans and A. Abarbanel, Eds. Academic Press, ch. 1, pp. 3-27, 1999.
[8] Conati C., Probabilistic assessment of user's emotions in educational games. Journal of Applied Artificial Intelligence, 16, 555-575, 2002.
[9] Conati C., How to evaluate models of user affect?. Proceedings of ADS 04, Tutorial and Research Workshop on Affective Dialogue Systems. Kloster Irsee, Germany, June 2004. p. 288-300, 2004.
[10] Craig, S.D., Graesser, A. C.,Sullins, J., & Gholson, B., Affect and learning: An exploratory look into the role of affect in learning. Journal of Educational Media, 29, 241-250, 2004.
[11] D'Mello, S.K., S.D. Craig, B. Gholson, S. Franklin, R.W. Picard, & A.C. Graesser, "Integrating Affect Sensors in an Intelligent Tutoring System." /In Affective Interactions: The Computer in the Affective Loop Workshop at 2005 International conference on Intelligent User Interfaces,/ AMC Press, New York, pp. 7-13, 2005.
[12] De Vicente, A., & Pain, H., Informing the detection of students' motivational state : An empirical study. In S.A. Cerri, G. Gouarderes, and F. Paraguacu (Eds Proceedings of the sixth international conference on intelligent tutoring systems (pp.933-943). Berlin, Germany: Springer, 2002.
[13] Fan, C., Sarrafzadeh, A., Overmyer, S., Hosseini, H. G., Biglari-Abhari, M., & Bigdeli, A. A fuzzy approach to facial expression analysis in intelligent tutoring systems. In Antonio Méndez-Vilas and J.A.Mesa Gonz├ílez(Eds.) Advances in Technology-based Education: Towards a Knowledge-based Society Vol 3. (pp. 1933-1937). Badajoz, Spain: Junta De Extremadura, 2003.
[14] Fisher R. A., The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179-188, 1936
[15] Harter, S., A new self-report scale of intrinsic versus extrinsic orientation in the classroom: Motivation and informational components. Developmental Psychology, 17, 300-312, 1981.
[16] Hastie T., A. Buja, & R. Tibshirani. Penalized discriminant analysis. Annals of Statistics, 23:73-102, 1995.
[17] Hastie T., R. Tibshirani, and A. Buja. Flexible discriminant analysis by optimal scoring. J. American Statistical Association, 89:1255-1270, 1994.
[18] Heraz, A., Razaki, R. & Frasson, C., Using machine learning to predict learner emotional state from brainwaves. 7th IEEE conference on Advanced Learning Technologies: ICALT 2007, Niigata, Japan, (In Press).
[19] Kort, B., Reilly, R., & Picard, R., An affective model of interplay between emotions and learning: Reengineering educational pedagogyÔÇö building a learning companion. In T. Okamoto, R. Hartley, Kinshuk, & J. P. Klus (Eds.), Proceedings IEEE International Conference on Advanced Learning Technology: Issues, Achievements and Challenges (pp. 43-48). Madison, Wisconsin: IEEE Computer Society, 2001.
[20] Lang, P. J., Behavioral treatment and bio-behavioral assessment: Computer applications. In J. B. Sidowski, J. H. Johnson, & T. A. Williams (Eds.), Technology in mental health care delivery) systems (pp. 119-137). Norwood, NJ: Ablex, 1980.
[21] Lang, P.J., Bradley, M.M., & Cuthbert, B.N. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-6. University of Florida, Gainesville, FL, 2005.
[22] Lepper, M. R., & Woolverton, M., The wisdom of practice: Lessons learned from the study of highly effective tutors. In J. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. 135-158). Orlando, FL: Academic Press, 2002.
[23] Lester, J. C., Towns, S.G. & FitzGerald, P.J., Achieving affective impact: visual emotive communication in lifelike pedagogical agents. The International Journal of Artificial Intelligence in Education, 10(3-4), 278-291, 1999.
[24] Litman, D. J., & Forbes-Riley, K. Predicting student emotions in computer-human tutoring dialogues. In Proceedings of the 42nd annual meeting of the association for computational linguistics (pp. 352-359). East Stroudsburg, PA: Association for Computational Linguistics, 2004.
[25] Mehrabian, A., & Russell, J. A., An approach to environmental psychology. Cambridge, MA: MIT Press, 1974.
[26] Miserandino, M., Children who do well in school: Individual differences in perceived competence and autonomy in above-average children. Journal of Educational Psychology, 88, 203-214, 1996.
[27] Picard, R. W., Affective computing. Cambridge: MIT Press, 1997.
[28] Quinlan, R., C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann Publishers, 1993.
[29] Robson C. Real word research: A resource for social scientist and practitioner researchers. Oxford: Blackwell, 1993.
[30] Snow, R., Corno, L., & Jackson, D., Individual differences in affective and cognitive functions. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 243-310). New York: Macmillan, 1996.
[31] Stipek, D., Motivation to Learn: From Theory to Practice 3rd edition. Boston: Allyn and Bacon, 1998.
[32] Tellegen, A., Structures of mood and personality and their relevance to assessing anxiety, with an emphasis on self-report. In A. H. Tuma & J. D. Maser (Eds.), Anxiety and the anxiety disorders (pp. 681-706). Hillsdale, NJ: Erlbaum, 1985.
[33] Wang, N., Johnson, W.L., Mayer, R., Rizzzo, P., Shaw, E., & Collins, H., The politeness effect: Pedagogical agents and learning gains. In Looi, C., McCalla, G., Bredeweg, B., & Breuker, J. (Eds.), Artificial intelligence in education (pp. 686ÔÇö693). Amsterdam: IOS Press, 2005.
[34] Witten, I.H., and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco, 2005.
[35] Wundt, W., Gundriss der Psychologie
[Outlines of psychology]. Leipzig, Germany: Entgelmann, 1896.
[36] Youden. W. J. How to evaluate accuracy. Materials Research and Standards, ASTM, 1961.