What/Why/When/Where/How Framework and Faculty Development Workshop to Improve the Utility of Narrative Evaluations for Assessing Internal Medicine Residents

Abstract

Introduction Clinical competency committees (CCCs) rely on narrative evaluations to assess resident competency. Despite the emphasis on these evaluations, their utility is frequently hindered by lack of sufficient detail for use by CCCs. Prior resources have sought to improve specificity of comments and use of evaluations by residents but not their utility for CCCs in assessing trainee performance. Methods We developed a 1-hour faculty development workshop focused on a newly devised framework for Department of Medicine faculty supervising internal medicine residents. The what/why/when/where/how framework highlighted key features of useful narrative evaluations: behaviors of strength and growth, contextualized observations, improvement over time, and actionable next steps. Workshop sessions were implemented at a large multisite internal medicine residency program. We assessed the workshop by measuring attendee confidence and skill in writing narrative evaluations useful for CCCs. Skill was assessed through a rubric adapted from literature on the utility of narrative evaluations. Results Fifty-four participants started the presurvey, and 33 completed the workshop, for a response rate of 61%. Participant confidence improved pre-, post-, and 3 months postworkshop. Total utility scores improved in mock evaluations from 12.4 to 15.5 and in real evaluations from 13.7 to 15.0, but only some subcomponent scores improved, with fewer improving in the real evaluations. Discussion A short workshop focusing on our framework improves confidence and utility of narrative evaluations of internal medicine residents for use by CCCs. Next steps should include developing more challenging components of narrative evaluations for continued improvement in trainee performance and faculty assessment.

Keywords

• Clinical Competency Committee
• Narrative Evaluations
• Assessment
• Competency-Based Medical Education (Competencies, Milestones, EPAs)
• Faculty Development
• Internal Medicine