GMS Zeitschrift für Medizinische Ausbildung (Jan 2005)
Selecting medical students at the University of Witten/Herdecke: Part I: Evaluation of inter-rater-reliability in the interview selection process
Abstract
Background: The University of Witten/Herdecke was founded in 1983 as the first private university in Germany to teach medicine. The University is not part of the centralized state system of medical placement distribution but rather has its own concept and procedure relating to its choice of students. In order to evaluate reliability and validity of the process of selecting medical students at the University of Witten/Herdecke, the first step was to analyse the interview selection process.Material and methods: The basic foundation of this examination were the results of the interview period of of the admission process from 1999 to 2000. In order to test inter-rater-reliability of the interviewers, a modified Cohen Kappa coefficient for multiple nominal scaled variables was used with regards to the group as a whole and also individually for the subgroups gender, admission year, age-groups and A Level results.Results: With Kappa-values ranging from 0.70 in 2000 to 0.88 in 1999 the reliability of the interviewer-process is obvious. However there are clear differences in the two years examined with regards to agreement in the decisions of interviewers. In the subgroup analysis with regards to gender no differences in the Kappa values were seen (male: 0,81; female: 0.8). Slightly less agreement can be seen with regards to subgroup analyses of A Level results of applicants ranging from 0.76 to 0.82.Discussion: In summary, this study shows that the application selection procedure for medicine at the medical faculty of the University of Witten/Herdecke is a reliable instrument with a high inter-rater reliability. This remains the case even though the interviewers do not seem to have standardized decision making criteria. Whether individual criteria/constellations were decisive with regards to the decision making process will be the subject of a further study.
