Evaluating the impact of faculty development programme initiative: are we really improving skills in MCQ writing?

Abstract

Objective: This study was done to evaluate the effectiveness of a 3-hour workshop in improving faculty competence in developing high quality test items. Methodology: The study took place in National University of Medical Sciences and its affiliated institutes in Pakistan, during a period of six months in year 2018. It was a descriptive, cross sectional study. A series of seven workshops on quality assurance in writing MCQs was conducted as part of faculty development program, emphasizing hand-on experience and prompt feedback. Participants’ satisfaction was evaluated with a post-workshop feedback questionnaire (Kirkpatrick 1). A self-made structured questionnaire was given as a pre-test and post-test, to check improvement in cognition, behaviour and item writing skills of faculty (Kirkpatrick 2). Paired t test was applied and difference in mean scores of responses was evaluated. Results: Total 141 faculty members were trained. The training session led to high satisfaction in all elements of workshop, significant improvements in boosting confidence in item writing skills(p=0.000), recognizing parts of MCQs (p=0.000), identifying item writing flaws (p=0.000) and levels of Millers pyramid and blooms taxonomy (p=0.000). Conclusion: Training sessions of short duration are effective in improving the competence of faculty in writing quality test items, provided hands-on experience is built-in and effective feedback is provided. Continuous...