Current Issues in Education (Aug 2013)
Criterion-Referenced Exit Examinations: An Institutionâs Internal Process for Psychometric Analysis
Abstract
The Texas State University-San Marcos undergraduate healthcare administration program requires all bachelors of health administration (BHA) students to pass a comprehensive examination to demonstrate their knowledge of specific core competencies. This also demonstrates completion of their didactic coursework in order to enter a practical internship or residency experience. Since this examination provided important documentation of student learning, the program conducted a detailed psychometric analysis of its three most recent undergraduate comprehensive exit examinations. In order to determine the value of this examination psychometrically, an evaluation of item validity evidence, between-exam reliability, and related assessment of descriptive statistics with regard to overall exam results and individual healthcare administration competency outcomes was necessary. Using Classical Test Theory (CTT) as a methodological framework, the psychometric analysis involved calculating item-level indices that assessed descriptive, validity, difficulty, and discrimination characteristics. This allowed the program’s faculty to better interpret student exam outcomes at the overall exam and within-exam competency levels. Additionally, this analysis provided an evaluation of the score reliability of the three alternate exam forms, as well as within-exam healthcare administration competency items, furthering the program’s comprehensive exit exam test development process. The outcomes of the analysis included an increased awareness of potential non-equivalent test forms for the total exam and within each exam (competency) level, increased level of interpretation the descriptive results for each exam, and the establishment of a more robust test development process to guide future comprehensive examination efforts.
