MedEdPublish (Dec 2018)
Making progress on identifying those who aren’t making progress: Using situational judgment tests to predict those at risk for remediation and attrition
Abstract
Introduction There is little evidence on tools that can be used to identify individuals most at risk for remediation and attrition in postgraduate surgical training. We explore the extent to which a situational judgment test (SJT) can predict which trainees are likely to require remediation or leave the program. Methods Postgraduate trainees in a single general surgery residency program in the United States completed a 50-item SJT. Data regarding remediation actions and attrition were retrieved from the program for the two years following completion of the assessment. United States Medical Licensing Examination (USMLE) scores were also included to examine their ability to predict remediation and attrition. Results Complete data was available from fifty-two of 56 (92%) residents in the program. Twenty-one percent (11/52) of residents required at least one remediation intervention within the one (for PGY5s) to two years after completing the SJT. Mann-Whitney U tests revealed a significant difference in SJT performance between those needing remediation versus those who did not require remediation, such that those requiring remediation performed worse on the assessment (p Conclusions These data demonstrate that SJTs can be created to effectively identify surgery residents most at risk for remediation across a two-year timeframe. These data provide an additional layer of validity evidence to support the role of SJTs in surgical education and align with other studies failing to find linkages between USMLE scores and residency performance criterion.
