A Distributed Cognition Framework to Compare E-Commerce Websites Using Data Envelopment Analysis

Authors: C. lo Storto

Abstract:

This paper presents an approach based on the adoption of a distributed cognition framework and a non parametric multicriteria evaluation methodology (DEA) designed specifically to compare e-commerce websites from the consumer/user viewpoint. In particular, the framework considers a website relative efficiency as a measure of its quality and usability. A website is modelled as a black box capable to provide the consumer/user with a set of functionalities. When the consumer/user interacts with the website to perform a task, he/she is involved in a cognitive activity, sustaining a cognitive cost to search, interpret and process information, and experiencing a sense of satisfaction. The degree of ambiguity and uncertainty he/she perceives and the needed search time determine the effort size – and, henceforth, the cognitive cost amount – he/she has to sustain to perform his/her task. On the contrary, task performing and result achievement induce a sense of gratification, satisfaction and usefulness. In total, 9 variables are measured, classified in a set of 3 website macro-dimensions (user experience, site navigability and structure). The framework is implemented to compare 40 websites of businesses performing electronic commerce in the information technology market. A questionnaire to collect subjective judgements for the websites in the sample was purposely designed and administered to 85 university students enrolled in computer science and information systems engineering undergraduate courses.

Keywords: Website, e-commerce, DEA, distributed cognition, evaluation, comparison.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1063016

References:

[1] R. Agarwal, and V. Venkatesh, "Assessing a firm's Web presence: A heuristic evaluation procedure for the measurement of usability", Information Systems Research, vol. 13, pp. 168-186, 2002.
[2] A.R. Andreasen, "A Taxonomy of Consumer Satisfaction/Dissatisfaction Measures", Journal of Consumer Affairs, vol. 11, n. 2, pp.11-24, 1977.
[3] R.D. Banker, A. Charnes, and W.W. Cooper, "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis", Management Science, vol. 30, n. 9, pp. 1078-1092, 1984.
[4] P. Barber, Applied Cognitive Psychology: An Information-Processing Framework, New York: Methuen, 1988.
[5] C. Bauer, and A. Scharl, "Quantitative evaluation of Web site content and structure", Internet Research: Electronic Networking Applications and Policy, vol. 10, n. 1, pp 31-43, 2000.
[6] W. Bearden, and J. Teel, "Selected determinants of consumer satisfaction and complaint reports", Journal of Marketing Research, vol. 20, pp. 21-28, 1983.
[7] J.R. Bettman, An Information Processing Theory of Consumer Choice, Addison-Wesley, Reading, MA, 1979.
[8] P.H. Bloch, D.L. Sherrell, and N.M. Ridgway, "Consumer search: an extended framework", Journal of Consumer Research, vol. 13, n. 1, pp. 119-126, 1986.
[9] J. Bloomfield, "Visual Search in Complex Fields: Size Differences Between Target Disc and Surrounding Discs", Human Factors, vol. 14, n. 2, pp. 139-148, 1972.
[10] E. Boling, "Usability testing for web sites", available on Internet http://www.indiana.edu/iirg/ARTICLES/usability/usability.main.htm, 1995.
[11] K.B. Bowen, "Website Evaluation: Experimental and Quasiexperimental Design Issues", available on Internet http://trochim.human.cornell.edu/webeval/webexper/webexper.htm, 1986.
[12] A. Boyd, "The goals, questions, indicators, measures (GQIM) approach to the measurement of customer satisfaction with e-commerce Web sites" Aslib Proceedings, vol 54, n. 3, pp. 177-187, 2002.
[13] J.M. Carroll, Interfacing Thought: Cognitive Aspects of Human- Computer Interaction, Cambridge, MA: The MIT Press, 1987.
[14] A. Charnes, W.W. Cooper, and E. Rhodes, "Measuring Efficiency of Decision Making Units", European Journal of Operations Research, vol. 2, n. 6, pp. 429-444, 1978.
[15] A. Clark, Being There: putting brain, body, and world together again, Cambridge, Mass., The MIT Press, 1997.
[16] G. Cooper, "Cognitive load theory as an aid for instructional design", Australian Journal of Educational Technology, vol. 6, n. 2, pp. 108-113, 1990.
[17] J. Cox, and B.G. Dale, "Key quality factors in Web site design and use: An examination", International Journal of Quality & Reliability Management, vol. 19, n. 7, pp 862-888, 2002.
[18] M.E. Crosby, M.K. Iding, and D.N. Chin, "Research on Task Complexity as a Foundation for Augmented Cognition", Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS-03), 2003.
[19] Cunliffe D., "Developing usable Web sites: A review and model", Internet Research: Electronic Networking Application and Policy, vol. 10, n. 4, pp 295-307, 2000.
[20] R.L. Daft, and R.H. Lengel, "Organizational Information Requirements, Media Richness and Structural Design", Management Science, vol. 32, pp. 554-571, 1986.
[21] R.L. Daft, and K.E. Weick, "Toward A Model of Organizations as Interpretation Systems", Academy of Management Review, vol. 9, pp. 284-295, 1984.
[22] A. Day, "A model for monitoring Web site effectiveness", Internet Research: Electronic Networking Applications and Policy, vol. 7, n. 2, pp. 1-9, 1997.
[23] G.M. Dran, P. Zhang, and R. Small, "Quality Web sites: an application of the Kano model to Web-site design", Proceedings of the Fifth Americas Conference on Information Systems, Milwaukee USA, 1999.
[24] C. Drury, M. Clement, and R. Clement, "The Effect of Area, Density, and Number of Background Characters on Visual Search", Human Factors, vol. 20, n. 5, pp. 597-602, 1978.
[25] K.D. Eason, "Towards the experimental study of usability", Behaviour and Information Technology, vol. 3, n. 2, 1984, pp. 133-143.
[26] R. East, Consumer Behavior: Advances and Applications in Marketing, London: Prentice Hall, 1997.
[27] S. Eilon, The Art of Reckoning: Analysis of Performance Criteria, London: Academic Press, 1984.
[28] J.F. Engel, R.D. Blackwell, and P.W. Miniard, Consumer Behavior, Fort Worth, TX: Dryden Press, 1995.
[29] S.E. Engel, and R.E. Granda, "Guidelines for Man/Display Interfaces", IBM Technical Report TR 00.2720, December, Poughkeepsie, NY: IBM, 1975.
[30] M.W. Eysenck, and M.T. Keane, Cognitive psychology: A Student-s Handbook (3rd ed.), Hove, UK: Erlbaum Taylor & Francis, 1995.
[31] M.J. Farrell, "The Measurement of Productive Efficiency", Journal of Royal Statistical Society, A(CXX), pp. 253-281, 1957.
[32] G.T. Fitzelle Jr., and W.M.K. Trochim, "Survey Evaluation of Web Site Instructional Technology: Does It Increase Student Learning?", available on Internet http://trochim.human.cornell.edu/webeval(webques/webques.htm, 1996.
[33] N.V. Flor, and E. Hutchins, "Analyzing Distributed Cognition in Software Teams: A Case Study of Collaborative Programming During Adaptive Software Maintenance", in J. Koenemann-Belliveau, T. Moher., and S. Robertson (eds.) Empirical Studies of Programmers: Fourth Workshop, Norwood, NJ: Ablex, 1992, pp. 36-64.
[34] J.R. Galbraith, Designing Complex Organizations, Reading, MA: Addison-Wesley, 1973.
[35] G. Gay, M. Stefanone, M. Grace-Martin, and H. Hembrooke, "The Effects of Wireless Computing in Collaborative Learning Environments", International Journal of Human-Computer Interaction, vol. 13, n. 2, pp. 257-276, 2001.
[36] P. Guriting, and N.O. Ndubisi, "Borneo online banking: evaluating customer perceptions and behavioural intention", Management Research News, vol. 29 n. 1/2, pp. 6-15, 2006.
[37] J.A. Howard, and J.N. Sheth, The theory of buyer behavior, New York: John Wiley & Sons, Inc, 1969.
[38] E. Hutchins, Cognition in the Wild, Cambridge: The MIT Press, 1995a.
[39] E. Hutchins, "How a cockpit remembers its speeds", Cognitive Science, vol. 19, pp. 126-189, 1995b.
[40] E. Hutchins, and T. Klausen, "Distributed Cognition in an Airline Cockpit", in D. Middleton, and Y. Engeström (eds.) Communication and Cognition at Work, Cambridge, Mass: Cambridge University Press, 1996, pp.15-54.
[41] S.L. Jarvenpaa, and P.A. Todd, "Consumer Reactions to Electronic Shopping on the World Wide Web", Journal of Electronic Commerce, vol. 1, n. 2, pp. 59-88, 1997.
[42] P.W. Jordan, An Introduction to usability, London: Taylor&Francis, 1998.
[43] J. Karat, "Evolving the scope of user-centered design", Comm. ACM, vol. 40, pp. 33-38, 1997.
[44] D. Kirsch, and P. Maglio, "On distinguishing epistemic from pragmatic action", Cognitive Science, vol. 18, pp. 513-549, 1994.
[45] D. Kirsh, "Adapting the Environment Instead of Oneself", Adaptive Behavior, vol. 4, n. 3/4, pp. 415-452, 1996.
[46] D. Kolb, Learning Styles Inventory, Boston: McBer & Co, 1984.
[47] P. Kotzé, "Defining and specifying graphs using formal model-based techniques", South African Computer Journal, vol. 26, pp. 217-221, 2000.
[48] S. Laforet, and X. Li, "Consumers- attitudes towards online and mobile banking in China", International Journal of Bank Marketing, vol. 23, n. 5, pp. 362-380, 2005.
[49] J. Lave, Cognition in practice: Mind, mathematics and culture in everyday life, Cambridge: Cambridge University Press, 1988.
[50] A. Lecerof, and F. Paterno, "Automatic support for usability evaluation", IEEE Transaction on Software Engineering, vol. 24, n. 10, pp. 863 - 888, 1998.
[51] P.H. Lindsay, and D.A. Norman, Human Information Processing: An Introduction to Psychology, 2nd ed., New York: Academic Press, 1977.
[52] G.L. Lohse, and P. Spiller, "Quantifying the effect of the user interface design features on cyberstore traffic and sales", CHI-98 Conference Proceedings, Los Alamitos, CA: ACM Press, 1998.
[53] E.T. Loiacono, "WebQual: a Web quality instrument", Proceedings of the Fifth Americas Conference on Information Systems, Milwaukee, USA, 1999.
[54] R. Merwe, and J. Bekker, "A framework and methodology for evaluating e-commerce Web sites", Internet Research: Electronic Networking Applications and Policy, vol. 13, n. 5, pp. 330-341, 2003.
[55] L. Mich, M. Franch, and L. Gaio, "Evaluating and designing Web site quality", IEEE MultiMedia, vol. 10, n. 1, pp 34-43, 2003.
[56] G.E. Miles, A. Howes, and A. Davies, "A framework for understanding human factors in web-based electronic commerce", International Journal of Human Computer Studies, vol. 52, pp. 131-163, 2000.
[57] G.A. Miller, "The magical number seven, plus or minus two: Some limits on our capacity for processing information", Psychological Review, vol. 63, pp. 81-97, 1956.
[58] S.Y. Mousavi, R. Low, and J. Sweller, "Reducing cognitive load by mixing auditory and visual presentation models", Journal of Educational Psychology, vol. 87, n. 2, pp. 319-334, 1995.
[59] A. Newell, and H. Simon, Human Problem Solving, Prentice Hall, 1972.
[60] F. Nicosia, Consumer Decision Processes: Marketing and Advertising Implications, Prentice Hall, 1966.
[61] J. Nielsen, Usability Engineering, Boston, MA: Academic Press, 1993.
[62] J. Nielsen, Multimedia and hypertext: the Internet and beyond, Boston: AP Professional, 1995.
[63] J. Nielsen, Designing Web Usability: The Practice of Simplicity, Indianapolis, IN: New Riders Publishing, 2000.
[64] J.W. Palmer, "Web site usability, design, and performance metrics", Information Systems Research, vol. 13, n. 2, pp 151-167, 2002.
[65] H.E. Pashler, The Psychology of Attention, Massachusetts: A Bradford Book, The MIT Press, 1998.
[66] N.L. Ping Zhang, "An assessment of human-computer interaction research in management information systems: topics and methods", Computers in Human Behavior, vol. 20, pp. 125-147, 2004.
[67] M. Polanyi, Personal Knowledge: Toward a Post-Critical Philosophy, Chicago, IL: University of Chicago Press, 1958.
[68] R. Radha, and C. Murphy, "Searching Versus Browsing in Hypertext", Hypermedia, vol. 4, n. 1, pp. 1-31, 1992.
[69] Y. Rogers, "New Theoretical approaches for Human-Computer Interaction", Annual Review of Information, Science and Technology, vol. 38, pp. 87-143, 2004.
[70] Y. Rogers, and J. Ellis, "Distributed Cognition: an alternative framework for analysing and explaining collaborative working", Journal of Information Technology, vol. 9, pp. 119-128, 1994.
[71] J.M. Scandura, "Deterministic theorizing in structural learning: Three levels of empiricism", Journal of Structural Learning, vol. 3, pp. 21-53, 1971.
[72] L. Schiffman, D. Bednall, E. Cowley, A. O-Cass, J. Watson, and L. Kanuk, Consumer behaviour (2nd ed). Australia: Pearson Education Australia Pty Ltd, 2001.
[73] H.A. Simon, "Rational decision-making in business organizations", American Economic Review, vol. 69, pp. 495-501, 1979.
[74] H.A. Simon, Models of My Life, Cambridge, MA: MIT Press, 1996.
[75] S. Singh, G. Erwin, and P. Kotzé, "Electronic Business Accepted Practices (e-BAP): Standardion of HCI for E-Commerce in South Africa", Technical Report, UNISA-TR-2001-22, 2001.
[76] R. Solso, Cognitive Psichology, Allyn & Bacon Editions, 1991.
[77] N. Staggers, and A.F. Norcio, "Mental models: concepts for human- computer interaction research", International Journal of Machine Studies, vol. 38, pp. 587-605, 1993.
[78] L.A. Suchman, Plans and situated actions: The problem of human computer interaction, Cambridge: Cambridge University Press, 1987.
[79] J. Sweller, "Cognitive load during problem solving: Effects on learning", Cognitive Science, vol. 12, pp. 257-285, 1988.
[80] J. Sweller, "Cognitive Load Theory, learning difficulty, and instructional design", Learning and Instruction, vol. 4, pp. 295-312, 1994.
[81] L.Y. Toh, and S. Pendse, "An empirical evaluation of Web page design and consumer behaviour", Proceedings of the 1st Annual Collected Workshop on Electronic Commerce, Adelaide Australia, 1997.
[82] A.M. Treisman, and C. Celade, A features integration theory of attention, Cognitive Psychology, vol. 12, pp. 96-136, 1980.
[83] W.M.K. Trochim, and D. Hover, "Mapping Student Views of the Benefits of a Course Website", available on Internet http://trochim.human.cornell.edu/webeval/mapuser/mapuser.htm, 1996.
[84] T.S. Tullis, "A method for evaluating Web page design concepts", Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Los Angeles USA, 1998.
[85] K. Weick, Sensemaking in Organizations, SAGE Publications, 1995.
[86] W.L. Wilkie, Consumer Behaviour, 3rd ed, John Wiley and Sons, 1994.
[87] M. Wolfinbarger, and M.C. Gilly, "ComQ: Dimensionalizing, measuring and predicting quality of the e-tail experience", Proceedings of the American Marketing Association Conference, February 22-25, vol.13, 2002.
[88] J. Zhang, "The Interaction of Internal and External Representations in a Problem Solving Task", Proceedings of the Thirteenth Annual Conference of Cognitive Science Society, Hillsdale, NJ: Erlbaum, 1991.
[89] J. Zhang, and Norman D.A., "Representations in distributed cognitive tasks", Cognitive Science, vol. 18, pp. 87-122, 1994.