Advances in Medical Education and Practice (Aug 2022)
A Generalizable Approach to Predicting Performance on USMLE Step 2 CK
Abstract
Jeffrey B Bird,1 Doreen M Olvet,1 Joanne M Willey,1 Judith M Brenner1,2 1Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, 11549, USA; 2Department of Medicine, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, 11549, USACorrespondence: Judith M Brenner, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 500 Hofstra University, Hempstead, NY, 11549-1000, USA, Tel +1 516 463-7590, Email [email protected]: The elimination of the USMLE Step 1 three-digit score has created a deficit in standardized performance metrics for undergraduate medical educators and residency program directors. It is likely that there will be greater emphasis on USMLE Step 2 CK, an exam found to be associated with later clinical performance in residents and physicians. Because many previous models relied on Step 1 scores to predict student performance on Step 2 CK, we developed a model using other metrics.Materials and Methods: Assessment data for 228 students in three cohorts (classes of 2018, 2019, and 2020) were collected, including the Medical College Admission Test (MCAT), NBME Customized Assessment Service (CAS) exams and NBME Subject exams. A linear regression model was conducted to predict Step 2 CK scores at five time-points: at the end of years one and two and at three trimester intervals in year three. An additional cohort (class of 2021) was used to validate the model.Results: Significant models were found at 5 time-points in the curriculum and increased in predictability as students progressed: end of year 1 (adj R2 = 0.29), end of year 2 (adj R2 = 0.34), clerkship trimester 1 (adj R2 = 0.52), clerkship trimester 2 (adj R2 = 0.58), clerkship trimester 3 (adj R2 = 0.62). Including Step 1 scores did not significantly improve the final model. Using metrics from the class of 2021, the model predicted Step 2 CK performance within a mean square error (MSE) of 8.3 points (SD = 6.8) at the end of year 1 increasing predictability incrementally to within a mean of 5.4 points (SD = 4.1) by the end of year 3.Conclusion: This model is highly generalizable and enables medical educators to predict student performance on Step 2 CK in the absence of Step 1 quantitative data as early as the end of the first year of medical education with increasingly stronger predictions as students progressed through the clerkship year.Keywords: assessment, prediction model, career advising, match, NBME
