The impact of UTeach observation protocol-based multiple-source feedback on improving the quality of online teaching in medical education

Abstract

Abstract Background The ability to teach, as a core competency, has always suffered from a lack of systematic cultivation. Objective This study aimed to explore the difference in teaching quality before and after the application of the UTOP (UTeach observation protocol)-based multiple source feedback (MSF) formative assessment of the quality improvement of medical education classrooms. Methods This study retrospectively analyzed the teaching quality of the Second Affiliated Hospital of Tianjin University of Traditional Chinese Medicine and the Third Affiliated Hospital of Chongqing Medical University. The participants were divided into control and UTOP groups according to the actual training they took in the teaching. All the participants, including teachers and trainee teachers, who were under conventional teaching methods were in the control group, and those who were under MSF training were in the UTOP group. UTOP and New Teacher Assessment Scores (NTAS) were evaluated, and the differences between the two courses were analyzed. The UTOP group was divided into a teacher group and a practice group according to differences in the participants’ teaching experience. The UTOP-MSF evaluation data were extracted and analyzed, and the differences in teaching quality were compared before and after the implementation of MSF training. Results The study was conducted for 10 months, with 43 participants completing 120 web-based lectures. In the control group, 32 participants were included, including 22 teachers with 27 courses and 10 trainee teachers who had no teaching experience with 13 courses. Eleven participants in the UTOP group received 9 rounds of MSF feedback training and completed 80 online lectures, including 3 teachers with 19 courses and 8 trainee teachers with 61 courses. A total of 324 evaluations from researchers were extracted and analyzed, including 162 UTOP scores and 162 NTAS; 575 MSF assessments were analyzed, including 438 from the teachers and 137 from the trainee teachers. The participants in the UTOP group had significantly higher UTOP and NTAS scores than did those in the control group did (59.84 ± 14.64 vs. 40.74 ± 6.80, p = 0.000; 73.00 (62.25, 80.00) vs. 53.00 (50.75, 57.00), p = 0.000, respectively). Among the participants who performed UTOP training, the UTOP-MSF scores increased for both teachers and trainee teachers (51.50 (37.75, 68.25) vs. 73.00 (68.00, 76.50), p = 0.022; 41.00 (36.00, 51.75) vs. 68.00 (59.75, 77.00), p = 0.000, respectively). The mean UTOP MSF scores were strongly positively correlated with the mean NTAS (r = 0.640, p = 0.000). Conclusion UTOP-based MSF training significantly improved teaching quality in medical education, and it can be an effective tool for evaluating teaching quality.

Keywords

• Multisource feedback
• UTeach observation protocol
• Medical education
• Medical teachers