Quality assurance procedures in assessment - a descriptive study of medical colleges in Pakistan

Abstract

Objective: Objective of the present study was to identify and describe the quality assurance procedures being implemented in assessment system of medical colleges in Pakistan. Methods: A cross-sectional study was conducted from March 2015 to December 2017 in 49 medical colleges of Pakistan, using mixed method technique. A semi-structured questionnaire was filled after informed consent. Data was analyzed by using SPSS version 21 (IBM). Results: In this study, 35(71.4%) of institutions followed a written assessment policy provided by the affiliated university, 09 (18%) never did so, 22 (44.8 %) participants reported that content experts checked if the questions matched the objectives, 17 (34.7%) reported that content expert would never/rarely check that. Majority, 42(85.7%) of institutions took strict steps to prevent cheating in exam, 26 (53.1%) institutions analyzed theory exam statistically. Discrimination index, difficulty index, reliability, and point biserial were calculated in 14(28.6%), 13(26.5%), 12(24.4%), and 7(14.3%) of the medical colleges respectively. Only 12(24.5%) of the institutions provided written feedback on the results, 15 (30.6%) institutions conducted internal audit annually. Themes belonging to assessment domain including, training for assessment, barriers and challenges, feedback and audit were identified. Conclusion: General issues related to quality assurance procedures in assessments (e.g. overall awareness about assessment policy) were in place in 60% of the colleges however, a large proportion did not have them. QA in assessments during exams was ensured by almost all medical colleges with only few exceptions. After exams QA was below average in terms of item analysis and feedback. Continuous...