Online test administration results in students selecting more responses to multiple-choice-multiple-response items

Abstract

We developed the Physics Inventory of Quantitative Literacy (PIQL) to assess students’ quantitative reasoning in introductory physics contexts. The PIQL includes several “multiple-choice-multiple-response” (MCMR) items (i.e., multiple-choice questions for which more than one response may be selected) as well as traditional single-response multiple-choice items. In this paper, we discuss differences in performance on MCMR items that seems to result from differences in administration method (paper versus online). In particular, we find a tendency for “clickiness” in online administration: students choose more responses to MCMR items when taking the electronic version of the assessment. Student performance on single-response multiple-choice items was not affected by administration method. These results suggest that MCMR items may provide a unique opportunity to probe differences in online and on-paper administration of low-stakes assessments.