A Hierarchical BERT-Based Transfer Learning Approach for Multi-Dimensional Essay Scoring

Abstract

The task of automated essay scoring (AES) continues to attract interdisciplinary attention due to its commercial and educational importance as well as related research challenges. Traditional AES approaches rely on handcrafted features, which are time-consuming and labor-intensive. Neural network approaches have recently given fantastic results in AES without feature engineering, but they usually require extensive annotated data. Moreover, most of the existing AES models only report a single holistic score without providing diagnostic information about various dimensions of writing quality. Focusing on these issues, we develop a novel approach using multi-task learning (MTL) with fine-tuning Bidirectional Encoder Representations from Transformers (BERT) for multi-dimensional AES tasks. As a state-of-the-art pre-trained language model, a BERT-based approach can improve AES tasks with limited training data. Meanwhile, we deal with long texts by proposing a hierarchical method and using the attention mechanism to automatically determine the contribution of different fractions of the input essay to the final score. For the multi-topic essay scoring tasks on the ASAP dataset, results reveal that our approach outperforms the average quadratic weighted Kappa (QWK) score by 4.5% compared with the strong baseline. We propose a self-collected dataset of Chinese EFL Learners’ Argumentation (CELA) to provide valuable information about writing quality from multiple rating dimensions, including holistic and five analytic scales. For the multi-rating dimensional essay scoring tasks on the CELA dataset, experimental results demonstrate that our model increases the average QWK score by 8.1% compared with the strong baseline.

Keywords

• Multi-dimensional essay scoring
• transfer learning
• BERT
• multi-task learning