Advances in Medical Education and Practice (Aug 2021)
Psychometric Properties of Visual Indicators of Teaching and Learning Success “VITALS” Instrument for Evaluation of Clinical Teachers
Abstract
Nada Al-Yousuf1, Salah Eldin Kassab2,3, Hasan Alsetri4, Hossam Hamdy51Department of Opthalmology, King Abdullah Medical City, Manama, Kingdom of Bahrain; 2Department of Physiology, Faculty of Medicine, Suez Canal University, Ismailia, Egypt; 3Department of Basic Medical Sciences, College of Medicine, Gulf Medical University, Ajman, United Arab Emirates; 4Department of Chemistry and Biochemistry, University of California, Los Angeles, CA, USA; 5Department of Surgery, Gulf Medical University, Ajman, United Arab EmiratesCorrespondence: Nada Al-YousufKing Abdullah Medical City, 61, King Abdulaziz Avenue, Manama, Kingdom of BahrainTel +973 77310071Fax +973 77310001Email [email protected]: We have previously developed an instrument for students’ evaluation of clinical teachers that we called Visual Indicators of Clinical Teaching and Learning Success (VITALS). This study measures the reliability of VITALS as an instrument for student evaluation of clinical tutors. Additionally, the study explores the minimum number of student raters necessary for an acceptable reliability, and provides evidence of construct validity of the evaluation scores.Materials and Methods: This retrospective study included 1825 evaluation forms completed by medical students evaluating clinical tutors using the VITALS instrument. Reliability was measured by applying generalizability theory (G-theory) analysis using a two-facet design (raters and items). A D-study was used to determine the minimum number of raters required to achieve a reliability ≥ 0.80. Face validity was tested by measuring tutors’ degree of agreement with the items of the study instrument.Results: The overall G-coefficient was 0.89. The subject of measurement (clinical tutors’ scores) represented 15.8% of the variance across all tutors and items. The variance due to the interaction between raters (students) and tutors contributed to 43.5%, while the variance due to items was negligible. The remaining 40% of the variance was due to unexplained sources of error. The D-study demonstrated that a minimum of 12 raters (students) are required to achieve a reliability of 0.80. Finally, most of the clinical tutors agreed that all items in the instrument were appropriate.Conclusion: We demonstrate that VITALS exhibits good psychometric properties. There should be at least 12 students rating each clinical tutor to have an acceptable level of reliability for the study instrument. Face validity of the study instrument was evidenced by its high level of approval among clinical tutors.Keywords: clinical teaching, tutors’ evaluation, reliability, validity, generalizability theory
