Evaluation of knowledge with multiple-choice tests: three of four options? Experience with admission examinations to medical and surgical postgraduate studies at University of Antioquia (Medellín, Colombia)

Abstract

Introduction: The aim of this study was to evaluate the effect of reducing the number of response options per question on the psychometric indicators of an exam for admission to postgraduate medical studies, at University of Antioquia, in Medellín, Colombia. Methodology: Application of psychometric assessment indexes from the perspective or two theories: the classical of measurement and the item response, to a test of 70 questions, applied in 2014 to 2.539 candidates. The least frequently chosen distractor was eliminated and randomly replaced by one of the three remaining ones. Results: Only 52.9% of the questions had three functional distractors. No difference was found in the difficulty, discrimination, standard error of measurement, Cronbach's alpha and the coefficient of biserial correlation (classical measurement theory). Also, there was no difference in the extent of item difficulty or ability of people (item response theory). The test with three options retained a good fit. Conclusion: Multiple choice tests with three response options performed as well as their four options counterparts.

Keywords

• Educational Measurement
• Item Analysis
• Multiple Choice Questions