Does Using None-of-the-Above (NOTA) Hurt Students’ Confidence?

Abstract

Students claim that multiple-choice questions can be tricky, particularly those with competitive incorrect choices or choices like none-of-the-above (NOTA). Additionally, assessment researchers suggest that using NOTA is problematic for assessment. In experiments conducted online (with trivia questions) and in the classroom (with course-related questions), I investigated the effects of including NOTA as a multiple-choice choice alternative on students’ confidence and performance. In four experiments, participants answered two types of questions: basic multiple-choice questions (basic condition) and equivalent questions in which one incorrect choice was replaced with NOTA (NOTA condition). Immediately after answering each question, participants rated their confidence in their answer to that question (item-by-item confidence). At the end of the experiments, participants made aggregate confidence judgments for the two types of questions and provided additional comments about the use of NOTA as an alternative. Surprisingly, I found no significant differences in item-by-item confidence or performance between the two conditions in any of the experiments. However, across all four experiments, when making aggregate judgments, participants provided lower confidence estimates in the NOTA condition than in the basic condition. Although people often report that NOTA questions hurt their confidence, the present results suggest that they might not—at least not on a question-by-question basis.

Keywords

• multiple-choice
• testing
• assessment
• learning
• metacognition
• confidence