Issues of practicality and impact in checklist-based scoring of writing

Abstract

In 2016, Euroexam International launched an extensive research campaign with the aims of increasing transparency and accountability while preserving our professional values and achievements. Reviewing and renewing our scoring methodology was uppermost on the agenda, especially as the diverse potential in checklist-based scoring was particularly appealing in our context. Research and development had to conform to the administrative frameworks including time and financial constraints, as well as recognise stakeholder needs. In the present study, we report on the issues we had to tackle when introducing checklist-based scoring into practice. The participants of this mixed-methods research were 12 examiners, six EFL teachers and 28 language learners in four study groups. We used a total of 600 scripts by stratified random sampling from live examinations at levels B2 and C1. In the quantitative study, two division schemes were trialled to decide how to share the workload between examiner pairs. In the qualitative inquiry, teachers and language learners were invited to share their views on checklist-based feedback for formative purposes. With mutually beneficial compromises that enabled the successful introduction of our checklists for writing, the most important implication of this study is that evolving stakeholder needs can be fulfilled with adequate flexibility.

Keywords

• practicality
• checklist-based scoring
• writing
• scoring methodology
• practical implementation
• english as a foreign language