Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia

Abstract

Abdullah Alkhayal,1 Shahla Aldhukair,2 Nahar Alselaim,1 Salah Aldekhayel,1 Sultan Alhabdan,1 Waleed Altaweel,3 Mohi Elden Magzoub,4 Mohammed Zamakhshary1,21Department of Surgery, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia; 2Public Health Section, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; 3Urology Department, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia; 4Department of Medical Education, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi ArabiaBackground: After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors’ perspective, and program directors’ attitudes toward the available objective tools to assess technical skills.Methods: This study was a cross-sectional survey of surgical program directors (PDs). The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs’ perspective and the PDs’ attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated.Results: Seventy-one (61%) participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative evaluation, whereas 15% perform only formative evaluations of their residents, and the remaining 22% conduct both summative and formative evaluations of their residents’ technical skills. Operative portfolios are kept by 53% of programs. The percentage of programs with mechanisms for remediation is 29% (19 of 65).Conclusion: The survey showed that surgical training programs use different tools to assess surgical skills competency. Having a clear remediation mechanism was highly associated with reporting remediation, which reflects the capability to detect struggling residents. Surgical training leadership should invest more in standardizing the assessment of surgical skills.Keywords: objective assessment, program directors, surgical skills, surgical residency