Faculty experience and reliability of assessing narrative reports using rubrics: Report from a dental school in India

Abstract

Background: Assessment drives learning is a well-known adage. If the assessment is subjective, it can under/overestimate the student's performance and thus impair the learning process. Rubrics help standardize the assessment by giving guidelines to both the assessor and the student and minimize subjectivity. Methodology: A mixed method design was employed to identify the inter- and intra-rater reliability of assessing narrative reports and explore the faculty perceptions on using rubrics for the assessment of narrative reports. Results: The inter-rater reliability scores were 0.08 without the use of rubrics and 0.93 with the use of rubrics demonstrating a statistically significant-excellent reliability when rubrics were used. The intra- and inter-examiner mean total assessment scores showed a statistically significant variation when rubrics were not used while the variation was reduced with the use of rubric. The two examiners who were part of the study welcomed the use of rubrics for assessment. They concurred that rubrics gave the assessor the schema and minimized the subjectivity of the assessment. Although the time taken for assessment was initially longer with a rubric scale, subsequently, the time taken was reduced. The assessors felt that they were now in a position to give richer constructive feedback to students. They felt that the students should also know the rubric scale beforehand so that it sets the expectations and helps the student to set self-determined learning goals. Conclusion: The use of rubrics improves inter- and intra-rater reliability of assessment scores. It gives a direction to the examiners and helps categorizing the student performance with minimal subjectivity.

Keywords

• examiner reliability
• performance assessment
• rubrics