A Mixed Methods Evaluation of Teaching Evaluation: Innovative Course-Based Service-Learning Model on Program Evaluation Competencies

Abstract

Abstract: There is limited empirical study examines effective evaluation pedagogy on evaluation competency outcomes. The purposes of this article are to: (1) identify key course design features critical to integrating real-world projects in a course-based service-learning model; and (2) provide empirical data using mixed methods evaluation to assess the impact on evaluation competencies. Data from 5 cohort doctoral students (2015-2019; n=51) showed such course design resulted in significantly increased before-after course scores on the 6-item course learning objectives (scale item means of 3.34 vs 4.65; p<.001), and the 17-item program evaluation competency scale (PECS-17) (scale item means of 3.25 vs 4.76; p<.001). Qualitative reflections on the 5 essential evaluation competency domains convergently were correlated with strong positive competency outcomes. The convergent findings from both quantitative and qualitative data provide strong empirical evidence of evaluation competencies gained. These findings have implications on teaching evaluation of graduate students for evaluator educators who strive to provide competency-based experiential learning.

Keywords

• course-based service-learning
• program evaluation competency
• mixed methods evaluation
• teaching evaluation
• competency assessment