Innovating Assessment: Exploring AI-Driven Scoring for Language Tests in Pre-Service Education Admissions

Authors: Lucie Bartosova

Abstract:

The rapid advancements in generative artificial intelligence (AI) have introduced transformative possibilities in education, particularly in assessment methodologies. This work provides an overview of the current state of the literature on AI-scoring methodologies for evaluating student-written responses. The focus is on how these innovations can be leveraged within large-scale assessments to address resource constraints such as limited assessors, time, and budget. Drawing from an initiative tied to a language test used for admitting candidates into a pre-service education program in the Faculty of Education at an Ontario university, the review explores the practical and ethical implications of integrating AI-driven tools into assessment processes. These tools are designed to automate the evaluation of learners’ written compositions, provide performance feedback, and support grading procedures. By synthesizing findings from recent research, the review highlights the effectiveness, reliability, and potential biases of AI in scoring, alongside considerations for transparency and fairness. This work emphasizes the dual role of generative AI as both a practical solution for scaling assessments and a subject of critical scrutiny to ensure its responsible implementation. The proposed integration of AI-scoring methodologies in our language test underscores the need to balance innovation with accountability, ensuring that AI tools enhance, rather than compromise, educational equity and rigor. OBJECTIVES OF YOUR RESEARCH To determine which generative AI model is most capable of evaluating written responses for university assessments based on specific criteria and to investigate potential biases within AI models to ensure fair assessments. METHODOLOGIES Evaluating generative AI models to determine their performance in assessing written responses against specific criteria. Collecting responses from previous assessments and annotating them with expert feedback to train and validate the AI models. MAIN CONTRIBUTIONS Introducing a tailored AI model to assess written responses on language tests. Offering a scalable and replicable model that informs broader applications of AI in educational assessments, contributing to policy-making and institutional best practices.

Keywords: artificial intelligence, assessment practices, student written performance, automated essay scoring, language proficiency