IJCM_380A: Shaping Medical Assessment with Expert Consensus: A Theory-Based: Summative Evaluation Blueprint for Community Medicine

Abstract

Background: Despite their constraints and failure to adhere to evaluation criteria, conventional written assessment methods are still widely and often used in many medical colleges. The partiality and inclination of the topic setter are apparent. This results in the overrepresentation of certain themes while leaving many others neglected. Every examiner possesses their own bias and tends to construct questions according to their specific areas of expertise. The distribution and arrangement of the questions are still ambiguous for learners. Several authors have suggested comprehensive evaluation frameworks for different levels of cognitive domains. Objective: This study sought to create a theory-based summative evaluation blueprint by involving experts from the All-India Institute of Medical Sciences. Methodology: Twenty-five experts from 10 All India Institutes of Medical Science were invited via email and contributed for summative assessment for a theory paper. Each week, the study lead team sent an online form containing 25 topics. A total of 269 topics were selected to be included in the blueprint development. An Impact and Frequency score was obtained for each topic, with a score between 1 and 3 for both its “Impact (I)” and “Frequency” (F). Results: The overall response rate was 71%. “Principle of epidemiology” (14.7%) had the most weight, followed by “Epidemiology of communicable disease” (13.4%) and “National Health Program” (10%). These three chapters constituted more than one-third of the total weight of the summative assessment (theory). 20 out of 24 (83%) chapters have moderate-to-good interrater reliability. Conclusion: The proposed blueprint will serve as a reference for setting the question paper and will impart objectivity, uniformity, and validity to the construction of the written assessment tool.

Keywords

• assessment
• undergraduate medical education