An E-Assessment Website to Implement Hierarchical Aggregate Assessment

Authors: M. Lesage, G. Raîche, M. Riopel, F. Fortin, D. Sebkhi

Abstract:

This paper describes a Web server implementation of the hierarchical aggregate assessment process in the field of education. This process describes itself as a field of teamwork assessment where teams can have multiple levels of hierarchy and supervision. This process is applied everywhere and is part of the management, education, assessment and computer science fields. The E-Assessment website named “Cluster” records in its database the students, the course material, the teams and the hierarchical relationships between the students. For the present research, the hierarchical relationships are team member, team leader and group administrator appointments. The group administrators have the responsibility to supervise team leaders. The experimentation of the application has been performed by high school students in geology courses and Canadian army cadets for navigation patrols in teams. This research extends the work of Nance that uses a hierarchical aggregation process similar as the one implemented in the “Cluster” application. 

Keywords: E-Learning, E-Assessment, Teamwork Assessment, Hierarchical Aggregate Assessment.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1091040

References:

[1] L. Allal, "Acquisition et Évaluation des Compétences en Milieu Scolaire” in J., Dolz and E., Ollagnier, Eds L’énigme de la compétence en éducation Bruxelles, de Boeck, 2002, pp. 77-94.
[2] D. P. Baker, and E. Salas Principles for Measuring Teamwork Skills. Human Factors, Vol. 34, 1992, pp. 469-475.
[3] Blackboard, 2013 (Online). Available: http://www.blackboard.com
[4] D. Boud, R. Cohen and J. Sampson, "Peer Learning and Assessment”, Assessment & Evaluation in Higher Education, Vol. 24, No. 4, 1999, pp. 413-426.
[5] J. P. Chin, V. A. Diehl, and K. L. Norman, "Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface”, In Proceedings of SIGCHI ‘88, New York. ACM/SIGCHI, 1988, pp. 213–218.
[6] C. C. Durham, D. Knight and E. A. Locke, "Effects of Leader Role, Team-Set Goal Difficulty, Efficacy, and Tactics on Team Effectiveness”, Organizational Behavior and Human Decision Process, Vol. 72, No. 2, 1997, pp. 203-231.
[7] M. Freeman, and J. McKenzie, "Self and Peer Assessment of Student Teamwork: Designing, Implementing and Evaluating SPARK, a Confidential, Web Based System” (Online), in Flexible Learning for a Flexible Society, Proceedings of ASET-HERDSA 2000 Conference, Toowoomba, Qld, 2-5 July, ASET and HERDSA, 2000. Access: http://www.aset.org.au/confs/aset-herdsa2000/procs/freeman.html.
[8] M. Freeman and J. McKenzie, "SPARK, a Confidential Web-Based Template for Self and Peer Assessment of Student Teamwork: Benefits of Evaluating across Different Subjects”, British Journal of Educational Technology, Vol. 33, No. 5, 2002, pp. 551-569.
[9] A. Furnham, H. Steele and D. Pendelton, "A Psychometric Assessment of the Belbin Team-Role Self-Perception Inventory”, Journal of Occupational and Organizational Psychology, Vol. 66, 1993, pp.245-257.
[10] J. Goldfinch, "Further Developments in Peer Assessment of Group Projects”, Assessment & Evaluation in Higher Education, Vol 19, No. 1, 1994, pp. 29-35.
[11] J. Goldfinch and R. Raeside, "Development of a Peer Assessment Technique for Obtaining Individual Marks on a Group Project”, Assessment & Evaluation in Higher Education, Vol.15, No. 3, 1990, pp. 210-231.
[12] D. Hart, "Authentic Assessment: A Handbook for Educators”, New York: Addison Wesley, 1994.
[13] W. Kaye, and M. E. Mancini, (1986), "Use of the Mega Code to Evaluate Team Leader Performance during Advanced Cardiac Life Support”, Critical Care Medecine, Vol. 14, No. 2, 1986, pp. 99-104.
[14] R. W. Lingard, "Teaching and Assessing Teamwork Skills in Engineering and Computer Science”, Journal of Systemics, Cybernetics and Informatics, Vol. 18, No. 1, 2010, pp.34-37.
[15] J. Loiselle, (2001), "La Recherche Développement en Éducation: sa Nature et ses Caractéristiques”, In M. Anadón and M. L'Hostie (Eds.), Nouvelles Dynamiques de Recherche en Éducation, Québec: Les Presses de l'Université Laval, 2001, pp. 77-97.
[16] R. Louis and H. Bernard, "L’évaluation des Apprentissages en Classe: Théorie et Pratique”, Montréal : Groupe Beauchemin, Éditeur Ltée, 2004, p. 79.
[17] S. J. Lurie, S. H. Schultz and G. Lamanna, "Assessing Teamwork: A Reliable Five-Question Survey”, Family Medicine, Vol. 43, No 10, 2011, pp. 731-734.
[18] J. MacDonald, "Assessing Online Collaborative Learning: Process and Product”, Computer & Education, 40, 2003, pp. 377-391.
[19] J. MacMillan, M. J. Paley, E. B. Entin, and E. E. Entin, "Questionnaires for Distributed Assessment of Team Mutual Awareness”, In N. A. Stanton, A. Hedge, K. Brookhuis, E. Salas, and H. W. Hendrick (Eds.), Handbook of Human Factors and Ergonomic Methods, Taylor and Francis, 2004.
[20] J. A. Marin-Garcia and J. Lloret, "Improving Teamwork with University Engineering Students: The effect of an Assessment Method to Prevent Shirking”, WSEAS Transactions on Advances in Engineering Education, Vol 1, No. 5, 2008, pp. 1-11.
[21] J. C. Marshall-Mies, E. A Fleishman, J. A. Martin, S. J. Zaccaro, W. A Baughman, and M. L. McGee, "Development and Evaluation of Cognitive and Metacognitive Measures for Predicting Leadership Potential”, Leadership Quaterly, Vol. 11, No. 1, 2000, pp.135-153.
[22] Moodle, 2013 (Online). Available: http://moodle.org
[23] W. D. Nance, "Improving Information Systems Students' Teamwork and Project Management Capabilities: Experiences from an Innovative Classroom”, Information Technology and Management, Vol. 1, No. 4, 2000, pp. 293-306.
[24] S. G. Northrup and D. A Northrup,. (2006) "Multidisciplinary Teamwork Assessment: Individual Contributions and Interdisciplinary Interaction”. Communication presented to the 36th ASEE/IEEE Frontiers Conference in Education, October 28–31, San Diego, 2006, p. 36.
[25] T. Palm,"Performance Assessment and Authentic Assessment: A Conceptual Analysis of the Literature” (Online). Practical Assessment, Research & Evaluation, Vol. 13, No. 4, 2008, p. 6. Access: http://pareonline.net/pdf/v13n4.pdf.
[26] G. Perlman, "Questionnaire for User Interface Satisfaction (Quis)”, 2012 (Online). Access: http://hcibib.org/perlman/ question.cgi?form=QUIS.
[27] P. Rennert-Ariev, "A Theoretical Model for the Authentic Assessment of Teaching” (Online), Practical Assessment, Research & Evaluation, Vol. 10, No. 2, 2005, p. 1. Access: http://pareonline.net/pdf/v10n2.pdf.
[28] R. C. Richey and W. A. Nelson, "Developmental Research”, In D. H. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology, New-York: Mac Millan, 1996, pp. 1213-1245.
[29] P. D Ritchie and P. A Cameron, "An Evaluation of Trauma Team Leader Performance by Video Recording”, Australian and New Zealand Journal of Surgery, Vol. 69, 1999, pp.183-186.
[30] D. F. Sittig, G. J Kuperman and J. Fiskio, "Evaluating Physician Satisfaction Regarding User Interactions with an Electronic Medical Record System”, Proceedings of the American Medical Informatics Association (AMIA) Annual Symposium, Vol. 400, No. 4, 1999.
[31] M. Sugrue, M. Seger, R. Kerridge, D. Sloane and S. Deane, "A Prospective Study of the Performance of the Trauma Team Leader”, The Journal of Trauma: Injury, Infection and Critical Care, Vol. 38, No. 1, 1995, pp.79-82.
[32] K. Swan, J. Shen and R. Hiltz, "Assessment and Collaboration in Online Learning”, Journal of Asynchronous Learning Networks, Vol. 10, p.1, 2006, pp. 45-62.
[33] J. Tardif, "L’évaluation des Compétences. Documenter le Parcours de Développement”, Montréal: Chenelière Éducation, 2006, p.125.
[34] A. Volkov and M. Volkov, "Teamwork and Assessment: A Critique”, e-Journal of Business Education & Scholarship of Teaching, Vol 1, 2007, pp.59-64.
[35] Webct, 2013 (Online). Available: http://www.webct.com.
[36] G. Wiggins, "Teaching to the (Authentic) Test”, Educational Leadership, Vol. 46, No. 7, 1989, pp. 41-47.
[37] G. Wiggins, "A True Test: Toward More Authentic and Equitable Assessment”, The Phi Delta Kappan, Vol. 70, No. 9, 1989, pp. 703-713.
[38] G. Wiggins, "The Case for Authentic Assessment” (Online). Practical Assessment, Research & Evaluation, Vol. 2, No. 2, 1990. Access: http://PAREonline.net/getvn.asp?v=2&n=2.
[39] G. Wiggins, "Assessment: Authenticity, Context and Validity”, The Phi Delta Kappan, Vol. 75, No. 3, 1993, pp. 200-214.
[40] Wikipedia., "Questionnaire for User Interaction Satisfaction (QUIS)” (Online), 2012. Access: http://en.wikipedia.org/wiki/ Questionnaire_for_User_Interaction_Satisfaction_(QUIS).
[41] K Willey and M. Freeman, "Completing the Learning Cycle: The Role of Formative Feedback When Using Self and Peer Assessment to Improve Teamwork and Engagement”, Proceedings of the 17th Annual Conference of the Australasian Association for Engineering Education, 10 -13 December 2006, Auckland, New Zealand, 2006.
[42] K., Willey and M. Freeman, "Improving Teamwork and Engagement: the Case for Self and Peer Assessment” (En Ligne), Australasian Journal of Engineering Education, 02, 2006. Access: http://www.aaee.com.au/journal/2006/willey0106.pdf.
[43] Worcester Polytechnic Institute (WPI), "Teamwork and Teamwork Assessment” (En ligne), 2011 Access: http://www.wpi.edu/Images/CMS/ IQP/teamworkandteamworkassessment.doc.
[44] S. J. Zaccaro, M. D. Mumford, M. S. Conelly, M. A. Marks and J. A Gilbert, "Assessment of Leader Problem-Solving Capabilities”, Leadership Quaterly, Vol. 11, No. 1, 2000, pp. 37-64.
[45] R.E. Clark, "Reconsidering Research on Learning from Media”, Review of Educational Research. Vol. 17, No. 2, 1983, pp. 445-459.
[46] R. B. Kozma, "Will Media Influence Learning: Reframing the Debate”, Educational Technology Research and Development (ETR&D), Vol. 42, No. 2, 1994, pp. 7-19.