Evaluating a Diagnostic Ankle Arthroscopy Scoring System to Assess Competency in Orthopaedic Trainees

Abstract

Category: Arthroscopy Introduction/Purpose: While there has been an increasing emphasis on assessing resident competency, little has been published on how to best evaluate competency for ankle arthroscopy. The purpose of this study was to design and validate an objective model for assessing basic ankle arthroscopy knowledge and surgical skills on a cadaveric ankle. Methods: The Basic Ankle Arthroscopy Skills Scoring System was adapted from previously validated assessment tools for knee arthroscopy. The scoring system includes (1) an oral questionnaire (0-23 points), (2) a surgical task-specific checklist (0-19 points), and (3) a global surgical skills rating (12-60 points). Ten medical students and twenty-three residents participated. All participants performed a diagnostic ankle arthroscopy on a cadaveric ankle and were assessed by a single observer. Six participants were tested by two evaluators to determine inter-observer reliability. Results: There was strong correlation between educational level and scores on the global surgical skills rating scale (r=0.967, p<0.0001), task-specific checklist (r=0.815, p<0.815), and oral questionnaire (r=0.896, p<0.0001). The global surgical skills scores were significantly different over multiple trainee levels but most notably between post graduate year (PGY) 1 and 2 (p<0.01) and between PGY 2 and 3 (p<0.05) [Figure 1a]. Oral questionnaire and task-specific checklists were significantly lower for medical students than PGY1 residents (p<0.001) [Figure 1b-c]. There was significant improvement in the oral questionnaire between senior and junior residents (p<0.05). There was a moderate correlation between number of ankle arthroscopy cases and scores on the global surgical skills score (r=0.7019, p<0.0001). Inter-observer reliability was high for the global surgical skills scores (ICC=0.89). Conclusion: The Basic Ankle Arthroscopy Skills Scoring System is a valid measure to objectively assess trainees’ ankle arthroscopy clinical knowledge and surgical skills in a bioskills laboratory. This tool will allow residency programs to evaluate competency and track individual progress over time.