A Simple Methodology for Discerning Item Construction Flaws in Health Professions Examinations

Abstract

Purpose: To introduce to health professions educators a simple methodology that can help discern item construction flaws and mitigate testwiseness effects. Method: The methodology involved administering a veterinary medical school exam to medical school professional staff participants with no formal training in the medical and health sciences. Results: The methodology was evidenced to be robust, as multiple items containing item construction flaws were identified by inflated success rates for a group of examinees who had no prior training in the subject matter. Discussion: Health professions educators are encouraged to utilize the methodology presented in this paper, where appropriate, to discern item construction flaws, reduce measurement error, and increase score validity relating to their assessments. Keywords: Medical education, Assessment, Psychometrics, Faculty Development, Grading, Health professions education