Comparing Student Performance on Paper-Based versus Computer-Based Formats of Standardized Tests

Authors: Jin Koo

Abstract:

During the coronavirus pandemic, there has been a further increasing demand for computer-based tests (CBT), and now it has become an important test mode. The main purpose of this study is to investigate the comparability of student scores obtained from computerized-based formats of a standardized test in the two subject areas of reading and mathematics. Also, this study investigates whether there is an interaction effect between test modes of CBT and paper-based tests (PBT) and gender/ability level in each subject area. The test used in this study is a multiple-choice standardized test for students in grades 8-11. For this study, data were collected during four test administrations: 2015-16, 2017-18, and 2020-21. This research used a one-factor between-subjects ANOVA to compute the PBT and CBT groups’ test means for each subject area (reading and mathematics). Also, 2-factor between-subjects ANOVAs were conducted to investigate examinee characteristics: gender (male and female), ethnicity (African-American, Asian, Hispanic, multi-racial, and White), and ability level (low, average, and high-ability groups). The author found that students’ test scores in the two subject areas varied across CBT and PBT by gender and ability level, meaning that gender, ethnicity, and ability level were related to the score difference. These results will be discussed according to the current testing systems. In addition, this study’s results will open up to school teachers and test developers the possible influence that gender, ethnicity, and ability level have on a student’s score based on whether they take the CBT or PBT.

Keywords: ability level, computer-based, gender, paper-based, test