Analytic assessment of summaries in LSP classes: The challenges and benefits involved

Abstract

Analytic assessment of discipline-specific summaries is a demanding task that may involve several criteria and more raters. Further impediments to the habitual use of analytic assessment of summaries in ESP classrooms are related to the development of reliable scoring rubrics, inter-rater agreement and objectivity of assessment. Wishing to outline possible solutions to assessment-related difficulties, and to gain more knowledge of our students’ summary writing abilities, we analysed summaries of a college-level text according to their content (factual accuracy, completeness, relevance and coherence) and text quality (cohesion and text organization). We discuss the assessment process (involving two raters), students’ summary writing performance and implications for teaching. Inter-rater reliability was satisfactory when assessing cohesion, text organisation, completeness and coherence, but was lower for relevance and factual accuracy. Students wrote accurate and fairly relevant summaries but they largely underperformed in coherence, cohesion and text organisation aspects of summary writing.

Keywords

• writing assessment
• summary writing
• analytic assessment
• ESP