Analytical Model Based Evaluation of Human Machine Interfaces Using Cognitive Modeling

Authors: Belkacem Chikhaoui, Helene Pigot

Abstract:

Cognitive models allow predicting some aspects of utility and usability of human machine interfaces (HMI), and simulating the interaction with these interfaces. The action of predicting is based on a task analysis, which investigates what a user is required to do in terms of actions and cognitive processes to achieve a task. Task analysis facilitates the understanding of the system-s functionalities. Cognitive models are part of the analytical approaches, that do not associate the users during the development process of the interface. This article presents a study about the evaluation of a human machine interaction with a contextual assistant-s interface using ACTR and GOMS cognitive models. The present work shows how these techniques may be applied in the evaluation of HMI, design and research by emphasizing firstly the task analysis and secondly the time execution of the task. In order to validate and support our results, an experimental study of user performance is conducted at the DOMUS laboratory, during the interaction with the contextual assistant-s interface. The results of our models show that the GOMS and ACT-R models give good and excellent predictions respectively of users performance at the task level, as well as the object level. Therefore, the simulated results are very close to the results obtained in the experimental study.

Keywords: HMI, interface evaluation, Analytical evaluation, cognitivemodeling, user modeling, user performance.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1334101

References:

[1] J. Nielsen and V. L. Phillips, "Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared," Proceedings of the INTERACT -93 and CHI -93 conference on Human factors in computing systems, pp. 214-221, 1993.
[2] B. D. Eugenio, S. Haller, and M. Glass, "Development and evaluation of nl interfaces in a small shop," AAAI Spring Symposium on Natural Language Generation in Spoken and Written Dialogue, pp. 1-8, 2003.
[3] B. Yen, P. Hu, and M. Wang, "Towards effective web site designs: A framework for modeling, design evaluation and enhancement," Proceedings of IEEE International Conference on e-Technology, e-Commerce, and e-Service (EEE05), pp. 1-6, 2005.
[4] L. M. Leventhal and J. A. Barnes, Usability Assessment, pearson prentice hall ed., ser. Usability Engineering: Process, Products, and Examples, 2007, ch. 11, pp. 206-220.
[5] P. M. Fitts, "The information capacity of the human motor system in controlling the amplitude of movement," Journal of Experimental Psychology, vol. 47, NO. 6, pp. 381-391, 1954.
[6] J. R. Anderson, D. Bothell, M. D. Byrne, S. Douglass, C. Lebiere, and Y. Qin, "An integrated theory of the mind," Psychological Review, vol. 111, 136-1060, 2004.
[7] B. E. John and D. E. Kieras, "Using goms for user interface design and evaluation: Which technique?" ACM Transactions on Computer-Human Interaction, vol. 3, pp. 287-319, 1996.
[8] H. Pigot, D. Lussier-Desrochers, J. Bauchet, Y. Lachapelle, and S. Giroux, "A smart home to assist recipes- completion." in Festival of International Conferences on Caregiving, Disability, Aging and Technology (FICCDAT), 2nd International Conference on Technology and Aging (ICTA), 2007.
[9] D. E. Kieras, "Goms models for task analysis," Handbook of task analysis for human-computer interaction, Lawrence Erlbaum Associates, no. Diaper D, Stanton N, pp. 83-116, 2003.
[10] A. Dix, J. Finlay, G. D. Abowd, and R. Beale, Human-Computer Interaction Third edition. England: Pearson, Prentice-Hall, Inc, 2004.
[11] J. Nielsen, Usability Engineering. Boston, MA: Academic Press, 1993.
[12] J. Preece, Y. Rogers, and H. Sharp, Introducing evaluation, ser. Interaction Design: Beyond Human-Computer Interaction. John Wiley and Sons, 2002, ch. 10, pp. 317-337.
[13] J. Nielsen and R. Molich, "Heuristic evaluation of user interfaces," in Proceedings of the SIGCHI conference on Human factors in computing systems. ACM New York, NY, USA, 1990, pp. 249-256.
[14] L.-O. Bligard and A.-L. Osvalder, An Analytical Approach for Predicting and Identifying Use Error and Usability Problem, a. holzinger. ed., ser. HCI and Usability for Medicine and Health Care, 2007, vol. 4799/2007, ch. 38, pp. 427-440.
[15] C. Stephanidis, D. Akoumianakis, M. Sfyrakis, and A. Paramythis, "Universal accessibility in hci: Process-oriented design guidelines and tool requirements," in Proceedings of the 4th ERCIM Workshop on User Interfaces for All, 1998, pp. 1-15.
[16] A. C. Siochi and D. Hix, "A study of computer-supported user interface evaluation using maximal repeating pattern analysis," in Proceedings of the SIGCHI conference on Human factors in computing systems, 1991, pp. 301-305.
[17] J. H. Goldberg and X. P. Kotval, "Computer interface evaluation using eye movements: methods and constructs," International Journal of Industrial Ergonomics, vol. 24, no. 6, pp. 631-645, 1999.
[18] R. S. Amant, T. E. Horton, and F. E. Ritter, "Model-based evaluation of expert cell phone menu interaction," ACM Transactions on Computer- Human Interaction, vol. 14(1), pp. 1-24, 2007.
[19] T. koskela, K. V.-V. Mattila, and L. Lehti, "Home is where your phone is: Usability evaluation of mobile phone ui for a smart home," in Proceedings of MobileHCI 2004, vol. 3160, 2004, pp. 74-85.
[20] B. E. John and D. D. Salvucci, "Multi-purpose prototypes for assessing user interfaces in pervasive computing systems," IEEE pervasive computing, vol. 4, no4, pp. 27-34, 2005.
[21] B. Chikhaoui and H. Pigot, "Simulation of a human machine interaction: Locate objects using a contextual assistant," in procedeengs of the 1st International North American Simulation Technology Conference, M. Beldjehem, Ed., 2008, pp. 75-80.
[22] P. Antunes, M. R. S. Borges, J. A. Pino, and L. Carrio, Analytic Evaluation of Groupware Design, ser. Computer Supported Cooperative Work in Design II, 2006, vol. 3865/2006, ch. 4, pp. 31-40.
[23] B. Chikhaoui and H. Pigot, "Evaluation of a contextual assistant interface using cognitive models," in procedeengs of the 5th International Conference on Human-Computer Interaction, waset, Ed., vol. 34, 2008, pp. 36-43.
[24] D. D. Salvucci and K. L. Macuga, "Predicting the effects of cellularphone dialing on driver performance," Cognitive Systems Research, vol. 3, no. 1, pp. 95-102, 2002.
[25] B. E. John and D. E. Kieras, "The goms family of analysis techniques: Tools for design and evaluation," Carnegie Mellon University School of Computer Science, Technical Report CMU-CS-94-181, 1994.
[26] A. Newell, Unified Theories of Cognition. Cambridge (Mass): Harvard University Press, 1990.
[27] J. R. Anderson, N. A. Taatgen, and M. D. Byrne, "Learning to achieve perfect time sharing: Architectural implications of hazeltine, teague ivry (2002)," Journal of Experimental Psychology: Human Perception and Performance, vol. 31, No. 4, pp. 749-761, 2005.
[28] M. D. Byrne, "Act-r/pm and menu selection: Applying a cognitive architecture to hci," International Journal of Human-Computer Studies, vol. 55, pp. 41-84, 2001.
[29] D. Bothell, "Act-r 6.0 reference manual," Working Draft, 2004.
[30] H. Pigot, J. Bauchet, and S. Giroux, Assistive devices for people with cognitive impairments, ser. The Engineering Handbook on Smart Technology for Aging, Disability and Independence. John Wiley and Sons, 2007, ch. 12.
[31] D. Lussier-Desrochers, Y. Lachapelle, H. Pigot, and J. Bauchet, "Apartments for people with intellectual disability: Promoting innovative community living services," in Proceedings of the 2nd International Conference on Intellectual Disabilities/Mental Retardation., 2007.
[32] B. E. John and D. E. Kieras, "The goms family of user interface analysis techniques: Comparison and contrast." ACM Transactions on Computer- Human Interaction, vol. 3, pp. 320-351, 1996.
[33] D. E. Kieras, S. D. Wood, K. Abotel, and A. Hornof, "Glean: A computer-based tool for rapid goms model usability evaluation of user interface designs," Proceedings of the 8th ACM Symposium on User Interface Software and Technology, no. ACM, pp. 91-100, 1996.
[34] A. Serna, H. Pigot, and V. Rialle, "Modeling the performances of persons suffering alzheimers disease on an activity of the daily living," in 18th Congress of the International Association of Gerontology, 2005.
[35] A. Dion and H. Pigot, "Modeling cognitive errors in the realization of an activity of the everyday life," in Cognitio, 2007.