Group versus modified individual standard-setting on multiple-choice questions with the Angoff method for fourth-year medical students in the internal medicine clerkship

Abstract

Vichai Senthong,1,* Jarin Chindaprasirt,1,* Kittisak Sawanyawisuth,1 Noppadol Aekphachaisawat,2 Suteeraporn Chaowattanapanit,1 Panita Limpawattana,1 Charoen Choonhakarn,1 Aumkhae Sookprasert1 1Department of Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; 2Central Library, Silpakorn University, Bangkok, Thailand *These authors contributed equally to this work Background: The Angoff method is one of the preferred methods for setting a passing level in an exam. Normally, group meetings are required, which may be a problem for busy medical educators. Here, we compared a modified Angoff individual method to the conventional group method. Methods: Six clinical instructors were divided into two groups matched by teaching experience: modified Angoff individual method (three persons) and conventional group method (three persons). The passing scores were set by using the Angoff theory. The groups set the scores individually and then met to determine the passing score. In the modified Angoff individual method, passing scores were judged by each instructor and the final passing score was adjusted by the concordance method and reliability index. Results: There were 94 fourth-year medical students who took the test. The mean (standard deviation) test score was 65.35 (8.38), with a median of 64 (range 46–82). The three individual instructors took 45, 60, and 60 minutes to finish the task, while the group spent 90 minutes in discussion. The final passing score in the modified Angoff individual method was 52.18 (56.75 minus 4.57) or 52 versus 51 from the standard group method. There was not much difference in numbers of failed students by either method (four versus three). Conclusion: The modified Angoff individual method may be a feasible way to set a standard passing score with less time consumed and more independent rather than group work by instructors. Keywords: Angoff, individual, passing score, standard-setting, multiple-choice questions, internal medicine