Automotive Emotions: An Investigation of Their Natures, Frequencies of Occurrence and Causes

Authors: Marlene Weber, Joseph Giacomin, Alessio Malizia, Lee Skrypchuk, Voula Gkatzidou

Abstract:

Technological and sociological developments in the automotive sector are shifting the focus of design towards developing a better understanding of driver needs, desires and emotions. Human centred design methods are being more frequently applied to automotive research, including the use of systems to detect human emotions in real-time. One method for a non-contact measurement of emotion with low intrusiveness is Facial-Expression Analysis (FEA). This paper describes a research study investigating emotional responses of 22 participants in a naturalistic driving environment by applying a multi-method approach. The research explored the possibility to investigate emotional responses and their frequencies during naturalistic driving through real-time FEA. Observational analysis was conducted to assign causes to the collected emotional responses. In total, 730 emotional responses were measured in the collective study time of 440 minutes. Causes were assigned to 92% of the measured emotional responses. This research establishes and validates a methodology for the study of emotions and their causes in the driving environment through which systems and factors causing positive and negative emotional effects can be identified.

Keywords: Affective computing, case study, emotion recognition, human computer interaction.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1474557

References:

[1] Gao, P, Kaas, H-W, Mohr, D & Wee, D 2016, Automotive revolution: perspective towards 2030: how the convergence of disruptive technology-driven trends could transform the auto industry, McKinsey and Company, viewed 11 Jan 2017, .
[2] KMPG 2016, The Rise of Autotech: Four emerging segments disrupting the auto industry, viewed 11 Jan 2017, https://home.kpmg.com/uk/en/home/insights/2016/04/the-rise-of-autotech-4-emerging-segments-disrupting-the-automoti.html
[3] Miller, D. (2001); Driven Societies, in D. Miller (ed.) Car Cultures. Oxford: Berg.
[4] Gkatzidou, V., Giacomin, J. and Skrypchuk. L. (2016). Automotive Habitat Laboratory: a facility for automotive co-design.Proceedings of the 7th International Conference on Applied Human Factors and Ergonomics, Orlando, Florida, USA. July 27-31.
[5] Giacomin, J. (2014). What Is Human Centred Design?. The Design Journal, 17(4), 606-623.
[6] Picard, Rosalind W. "Affective computing: from laughter to IEEE." IEEE Transactions on Affective Computing 1, no. 1 (2010): 11-17.
[7] Normark, Carl Jörgen, and Dimitrios Gkouskos. "Exploring user needs in automobiles." In International Design Conference: 21/05/2012-24/05/2012, pp. 1369-1376. Design Research Society, 2012.
[8] Healey (2000), “Wearable and Automotive Systems for Affect Recognition from Physiology,” Ph.D. Dissertation.
[9] Jones and I. M. Jonsson (2005), Automatic recognition of affective cues in the speech of car drivers to allow appropriate responses, in Proceedings of the 17th Australia Conference on Computer-Human Interaction: Citizens Online (OZCHI '05), vol.122, pp. 1–10, Canberra, Australia.
[10] Hoch, Stefan, et al. "Bimodal fusion of emotional data in an automotive environment." Proceedings. (ICASSP'05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005. Vol. 2. IEEE, 2005.
[11] Eyben, Florian, et al. "Emotion on the road—necessity, acceptance, and feasibility of affective computing in the car." Advances in human-computer interaction 2010
[12] Wells-Parker, J. Ceminsky, and J. Ceminsky, (2002) An exploratory study of the relationship between road rage and crash experience in a representative sample of US drivers, Accident Analysis and Prevention, vol. 34, no. 3, pp. 271–278.
[13] Dula, Chris S., and Mary E. Ballard. "Development and evaluation of a measure of dangerous, aggressive, negative emotional, and risky driving." Journal of Applied Social Psychology 33, no. 2 (2003): 263-282.
[14] Lee, Boon Giin, Teak Wei Chong, Boon Leng Lee, Hee Joon Park, Yoon Nyun Kim, and Beomjoon Kim. "Wearable Mobile-Based Emotional Response-Monitoring System for Drivers." IEEE Transactions on Human-Machine Systems (2017).
[15] Lisetti, Christine L., and Fatma Nasoz. "Affective intelligent car interfaces with emotion recognition." Proceedings of 11th International Conference on Human Computer Interaction, Las Vegas, NV, USA. 2005.
[16] Huguenin, Raphael D., and Kare Rumar. "Models in traffic psychology." Traffic psychology today. Springer US, 2001. 31-59.
[17] Mesken, Jolieke. Determinants and consequences of drivers' emotions. Stichting Wetenschappelijk Onderzoek Verkeersveiligheid SWOV, 2006.
[18] Healey, Jennifer Anne. Wearable and automotive systems for affect recognition from physiology. Diss. Massachusetts Institute of Technology, 2000.
[19] Busso, Carlos, et al. "Analysis of emotion recognition using facial expressions, speech and multimodal information." Proceedings of the 6th international conference on Multimodal interfaces. ACM, 2004.
[20] Parkinson, Brian. "Anger on and off the road." British journal of Psychology 92, no. 3 (2001): 507-526.
[21] Underwood, Geoffrey, Peter Chapman, Sharon Wright, and David Crundall. "Anger while driving." Transportation Research Part F: Traffic Psychology and Behaviour 2, no. 1 (1999): 55-68.
[22] Levelt, P. B. M. "Praktijkstudie naar emoties in het verkeer." S WOV Report R-2003-08. Leidschendam: SWOV (2003).
[23] Ekman, Paul, Wallace V. Friesen, and J. Hager. "The Facial Action Coding System (FACS): A technique for the measurement of facial action. Palo Alto." CA: Consulting Psychologists Press, Inc. Ekman, P. Levenson. RW, & Friesen WV (1983). Auto-nomic nervous system activity distinguishes among emotions. Science221 (1978): 1208-12.
[24] McDuff, Daniel, Abdelrahman Mahmoud, Mohammad Mavadati, May Amr, Jay Turcot, and Rana el Kaliouby. "AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit." In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3723-3726. ACM, 2016.
[25] Tian, Y-I., Takeo Kanade, and Jeffrey F. Cohn. "Recognizing action units for facial expression analysis." IEEE Transactions on pattern analysis and machine intelligence 23, no. 2 (2001): 97-115.
[26] Shah, Miraj, David G. Cooper, Houwei Cao, Ruben C. Gur, Ani Nenkova, and Ragini Verma. "Action unit models of facial expression of emotion in the presence of speech." In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on, pp. 49-54. IEEE, 2013.
[27] Namba, Shushi, Russell S. Kabir, Makoto Miyatani, and Takashi Nakao. "Spontaneous Facial Actions Map onto Emotional Experiences in a Non-social Context: Toward a Component-Based Approach." Frontiers in Psychology 8 (2017).
[28] Velusamy, Sudha, Hariprasad Kannan, Balasubramanian Anand, Anshul Sharma, and Bilva Navathe. "A method to infer emotions from facial action units." In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 2028-2031. IEEE, 2011.
[29] Wegrzyn, Martin, Maria Vogt, Berna Kireclioglu, Julia Schneider, and Johanna Kissler. "Mapping the emotional face. How individual face parts contribute to successful emotion recognition." PloS one12, no. 5 (2017): e0177239.
[30] Cienki, Alan, and Irene Mittelberg. "Creativity in the forms and functions of spontaneous gestures with speech." The Agile Mind: A Multidisciplinary Study of a Multifaceted Phenomenon. Berlin, Germany: De Gruyter Mouton (2013): 231-252.
[31] Duenwald, Mary. "The physiology of facial expressions." Retrieved September 19 (2005): 2007.
[32] Elliott, Eeva A., and Arthur M. Jacobs. "Facial expressions, emotions, and sign languages." Frontiers in psychology 4 (2013).
[33] Russell, James A., and José Miguel Fernández-Dols, eds. The psychology of facial expression. Cambridge university press, 1997.
[34] iMotions 2013. “Attention Tool” Retrieved September 25, 2015 from https://imotions.com/blog/attention-tool/.
[35] Regan, M. A., Aa Williamson, Ra Grzebieta, J. Charlton, M. Lenne, B. Watson, Nc Haworth et al. "The Australian 400-car naturalistic driving study: Innovation in road safety research and policy." In Proceedings of the 2013 Australasian road safety research, policing & education conference, Brisbane, Queensland. 2013.
[36] Barnard, Yvonne, Fabian Utesch, Nicole van Nes, Rob Eenink, and Martin Baumann. "The study design of UDRIVE: the naturalistic driving study across Europe for cars, trucks and scooters." European Transport Research Review 8, no. 2 (2016): 14.
[37] Klauer, Sheila G., Vicki L. Neale, Thomas A. Dingus, David Ramsey, and Jeremy Sudweeks. "Driver inattention: A contributing factor to crashes and near-crashes." In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 49, no. 22, pp. 1922-1926. Sage CA: Los Angeles, CA: SAGE Publications, 2005.
[38] Healey, Jennifer A., and Rosalind W. Picard. "Detecting stress during real-world driving tasks using physiological sensors." IEEE Transactions on intelligent transportation systems 6, no. 2 (2005): 156-166.
[39] Giacomin, J., and R. Bracco. "An experimental approach for the vibration optimisation of automotive seats, ATA 3rd Int." In Conf. on Vehicle Comfort and Ergonomics, Bologna, Italy, March, pp. 29-31. 1995.
[40] Map retrieved December 12, 2017 from https://app.traveltimeplatform.com
[41] Turner, Cathy, and Rod McClure. "Age and gender differences in risk-taking behaviour as an explanation for high incidence of motor vehicle crashes as a driver in young males." Injury control and safety promotion 10, no. 3 (2003): 123-130.
[42] Creswell, John W., and Cheryl N. Poth. Qualitative inquiry and research design: Choosing among five approaches. Sage publications, 2017.
[43] Gkouskos, Dimitrios, Carl Jörgen Normark, and Sus Lundgren. "What drivers really want: Investigating dimensions in automobile user needs." International Journal of Design 8, no. 1 (2014).