Towards automated content analysis of educational feedback: A multi-language study

Abstract

Feedback is a crucial element of a student's learning process. It enables students to identify weaknesses and improve self-regulation. However, studies show this to be an area of great dissatisfaction in higher education. With ever-growing course participation numbers, delivering effective feedback is becoming an increasingly challenging task. The efficacy of feedback will depend on four levels of feedback; namely, feedback about the self, task, process or self-regulation. Hence, this paper explores the use of automated content analysis to examine feedback provided by instructors for feedback practices measured on self, task, process, and self-regulation levels. For this purpose, four binary XGBoost classifiers were trained and evaluated, one for each level of feedback. The results indicate effective classification performance on self, task, and process levels with accuracy values of 0.87, 0.82, and 0.69, respectively. Additionally, inter-language transferability of feedback features is measured using cross-language classification performance and feature importance analysis. Findings indicate a low generalizability of features between English and Portuguese feedback spaces.

Keywords

• Natural language processing
• Classification
• Feedback
• Content analysis