Investigating Student Behavior in Adopting Online Formative Assessment Feedback

Authors: Peter Clutterbuck, Terry Rowlands, Owen Seamons

Abstract:

In this paper we describe one critical research program within a complex, ongoing multi-year project (2010 to 2014 inclusive) with the overall goal to improve the learning outcomes for first year undergraduate commerce/business students within an Information Systems (IS) subject with very large enrolment. The single research program described in this paper is the analysis of student attitudes and decision making in relation to the availability of formative assessment feedback via Web-based real time conferencing and document exchange software (Adobe Connect). The formative assessment feedback between teaching staff and students is in respect of an authentic problem-based, team-completed assignment. The analysis of student attitudes and decision making is investigated via both qualitative (firstly) and quantitative (secondly) application of the Theory of Planned Behavior (TPB) with a two statistically-significant and separate trial samples of the enrolled students. The initial qualitative TPB investigation revealed that perceived self-efficacy, improved time-management, and lecturer-student relationship building were the major factors in shaping an overall favorable student attitude to online feedback, whilst some students expressed valid concerns with perceived control limitations identified within the online feedback protocols. The subsequent quantitative TPB investigation then confirmed that attitude towards usage, subjective norms surrounding usage, and perceived behavioral control of usage were all significant in shaping student intention to use the online feedback protocol, with these three variables explaining 63 percent of the variance in the behavioral intention to use the online feedback protocol. The identification in this research of perceived behavioral control as a significant determinant in student usage of a specific technology component within a virtual learning environment (VLE) suggests that VLEs could now be viewed not as a single, atomic entity, but as a spectrum of technology offerings ranging from the mature and simple (e.g., email, Web downloads) to the cutting-edge and challenging (e.g., Web conferencing and real-time document exchange). That is, that all VLEs should not be considered the same. The results of this research suggest that tertiary students have the technological sophistication to assess a VLE in this more selective manner.

Keywords: Formative assessment feedback, virtual learning environment, theory of planned behavior, perceived behavioral control.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1099344

References:

[1] Freebody, P., & Luke, A. “Literacies programs: Debates and demands in cultural context.” Prospect: Australian Journal of TESOL, 5(7), 1990, pp. 7-16.
[2] Freebody, P., & Luke, A. “Further notes on the Four Resources Model”. 1999 Available online at: http: //www.readingonline.org/research/ lukefreebody.html (last accessed 1st September 2010).
[3] Clutterbuck, P., Seamons, O., & Rowlands, T. “Enhancing Information Systems Literacy Education via the Four Resources Model”. Proceedings of the Sixth Education in a Changing Environment (ECE) Conference, Salford. 2011, UK.
[4] Clutterbuck, P., Seamons, O., & Rowlands, T. “Enhancing Information Systems Literacy Education via the Four Resources Literacy Education Model – Further Analysis”, International Journal of Arts & Sciences, 5(2), 2012, pp. 177-194.
[5] Wilson, B. G. “Constructivist Learning Environments: Case Studies in Instructional Design”, Educational Technology Publications, 1996, Englewood Cliffs, NJ.
[6] Piccoli, G., Ahmad, R., Ives, B. “Web-Based Virtual Learning Environments: A Research Framework and a Preliminary Assessment of Effectiveness in Basic IT Skills Training”, MIS Quarterly, Vol. 25. No. 4, 2001, pp. 402-426.
[7] Keeves, J. P. (1994). “Assessment in schools, methods of assessment”. In Husen, Torsten, Postlethwaite, & T. Neville (Eds.), 2nd ed, The international encyclopedia of education, vol. 1, 1994, pp. 362-370. Oxford: Pergamon Press.
[8] Reeves, T. C., & Hedberg, J. G. “Evaluation strategies for open and distributed learning environments”. In C. Spratt, & P. Lajbcygier (Eds.), E-Learning technologies and evidence based assessment approaches, 2009, pp. 234-253, New York: Information Science Reference.
[9] Oosterhof, A., Conrad, R. M., & Ely, D. P. “Assessing learners online”, 2008, New Jersey: Pearson.
[10] Hargreaves, E. “Assessment”. In G. McCulloch, & D. Crook (Eds.), The Routledge international encyclopedia of education, 2008, pp. 37-38, New York: Routledge.
[11] Vonderwell, S., Liang, X., & Alderman, K. “Asynchronous discussions and assessment in online learning”. Journal of Research on Technology in Education, 39(3), 2007, pp. 309-328.
[12] Black, P., & Wiliam, D. “Developing the theory of formative assessment”. Educational Assessment, Evaluation and Accountability (formerly the Journal of Personnel Evaluation in Education), 21(1), 2009, pp. 5-31.
[13] Pachler, N., Daly, C., Mor, Y., & Mellar, H. “Formative e-assessment: Practitioner cases”. Computers & Education, 54, 2010, pp. 715-721.
[14] Gikandi, J. W., Morrow, D., & Davis, N. E. “Online formative assessment in higher education: A review of the literature”. Computers & Education, 57, 2011, pp. 2333-2351.
[15] Hyman, P. “In the Year of Disruptive Education”. Communications of the ACM. 55(12), 2012, 20-22.
[16] Kaufmann, P. B., & Mohan, J. “Video use and higher education: Options for the future”, 2009, New York University. Available at: http://library.nyu.edu/about/Video_Use_in_Higher_Education.pdf
[17] Moran, M., Seaman, J., & Tinti-Kane, H. “Teaching, learning, and sharing: How today’s higher education faculty use social media”, 2011, Available from: http://www.pearsonlearningsolutions.com/educators/ pearson-social-media-survey-2011-bw.pdf.
[18] Gardner, H. “Multiple intelligences”. New Horizons, 2006, New York: Basic Books.
[19] Corporation for Public Broadcasting. “Television goes to school: the impact of video on student learning in formal education”, 2004, Available from: http://www.dcmp.org/caai/nadh173.pdf.
[20] Holtzblatt, M., & Tschakert, N. “Expanding your accounting classroom with digital video technology”. Journal of Accounting Education, 29, 2011, pp. 100-121.
[21] Merriam, B. “Qualitative research and case study applications in education”, 1998, San Francisco: Jossey-Bass.
[22] Travers, M. “Qualitative Research through Case Studies”, 2001, Sage Publications, London.
[23] Marshall, C., Rossman, G. “Designing Qualitative Research”, 1989, Sage Publications, California.
[24] Glaser, B.G., Strauss, A.L. “The Discovery of grounded theory: strategies for qualitative research”. Aldine Publishing Company, 1967, Chicago.
[25] Ajzen, I. “The Theory of Planned Behavior”, Organizational Behavior and Human Decision Processes, Vol. 50, 1991, pp. 179-211.
[26] Kline, R.B. “Principles and Practice of Structural Equation Modeling, 2nd ed., 2005, Guilford Press, New York, NY.
[27] Fornell, C. and Larcker, D.F. “Evaluating structural equation models with unobservable variables and measurement error”, Journal of Marketing Research, Vol. 28, No. 1, 1981, pp. 39-50.
[28] Hair, J.F. Jr, Black, W.C., Babin, B.J., Anderson, R.E. and Tatham, R.L. “Multivariate Data Analysis, 6th ed., 2006, Prentice-Hall International, Englewood Cliffs, NJ.
[29] Nunnally, J.C. and Bernstein, I.H. “Psychometric Theory, 1994, McGraw-Hill, New York, NY.
[30] Armitage, C.J. and Conner, M. “Efficacy of the Theory of Planned Behavior, a meta-analysis review”, British Journal of Social Psychology, Vo. 40, No. 4, 2001, pp. 471-499.
[31] Compeau, D.R. and Higgins, C.A. “Computer self-efficacy: development of a measure and initial test”, MIS Quarterly, Vol. 19, No. 2, 1995, pp. 189-211.
[32] Teo, T., and Lee, C. B. “Explaining the intention to use technology among student teachers – an application of the Theory of Planned Behavior (TPB)”, Campus Wide Information Systems, 2010, Vol. 27, No. 2, pp. 60-67.
[33] Larreamendy-Joerns, J., & Leinhardt, G. “Going the distance with online education”, Review of Educational Research, 76(4), 2006, pp. 567-605.