Severity Differences across Proficiency Levels among Peer-assessors

Abstract

Over the past few years, peer-assessment, as an alternative assessment procedure, has drawn the attention of many researchers. In the study, it was attempted to find what kinds of language components peer-assessors attend to when rating their peers' essays and to investigate whether proficiency levels of peer-assessors make a difference in terms of severity and leniency they exercise. Fifty-eight student raters at Imam Khomeini International University in Qazvin rated five essays, using an analytic rating scale. Paper-based test of English as a foreign language (TOEFL) and five-paragraph essays were used to collect the data. FACETS (version 3.68.1) was used to analyze the data. The results of Facets analysis indicated that advanced peer-assessors had more variability in their severity compared to intermediate peer-assessors. Moreover, the majority of peer-assessors were, on average, more severe than lenient. The results also revealed no statistically significant difference between the ratings of intermediate and advanced peer-assessors. The final finding was that task achievement was the most attended assessment criterion, but grammatical range and accuracy was the least attended assessment criterion. The findings suggest peer-assessors do not attach an equal weight to all assessment criteria. The findings of the study may carry implications for the summative assessment of students' abilities.

Keywords

• criterion
• peer-assessment
• proficiency level
• severity