Developing and validating an online module for formative assessment of summary writing with automated content feedback for EFL academic writing instruction

Abstract

Abstract The present paper provides an overview of an online module for formative assessment of summary writing skills for second language (L2) introductory academic writing instruction in Japan and presents initial empirical results on how Japanese undergraduate students’ summary writing performance changed with a series of automated summary content feedback delivered in the module. A key feature of this module was the provision of fine-grained feedback delivered as scaffolding during revisions in terms of two key aspects of summary content: main idea representation and paraphrasing. Participants were 64 Japanese undergraduate engineering majors in introductory academic writing courses at a private university in Tokyo. The students completed two summary writing tasks provided through the online module. Results of a multivariate analysis of variance showed significant improvement of the content analytic score on revision on the initial summary task, and that this improved performance level was retained on a transfer task. The language use analytic score also improved significantly on the transfer task. Detailed analyses of learner-produced summaries based on descriptive statistics further suggested that the learners made substantively meaningful changes concerning main idea coverage and verbatim copying of the source text while still meeting the length requirement, although the results differed somewhat across the source texts assigned. Despite some study limitations, these results provide initial support for immediate content feedback provision for the development of basic summary writing skills.

Keywords

• Academic writing
• Summary writing
• Automated writing evaluation
• Content feedback
• Paraphrasing
• Formative assessment