Enhancing Feedback Quality at Scale: Leveraging Machine Learning for Learner-Centered Feedback

Abstract

In higher education, delivering effective feedback is pivotal for enhancing student learning but remains challenging due to the scale and diversity of student populations. Learner-centered feedback, a robust approach to effective feedback that tailors to individual student needs, encompasses three key dimensions—Future Impact, Sensemaking, and Agency, which collectively include eight specific components, thereby enhancing its relevance and impact in the learning process. However, providing consistent and effective learner-centered feedback at scale is challenging for educators. This study addresses this challenge by automating the analysis of feedback content to promote effective learner-centered feedback principles. We gathered a large-scale dataset of 16,531 feedback entries from 95 courses from one Master's and one Bachelor's program within a large faculty at an Australian university, with each entry labeled in accordance with the learner-centered feedback framework. Employing a range of machine learning and deep learning techniques, including Random Forest, XGBoost, BERT, and GPT-3.5, we systematically investigated the effectiveness of different approaches for constructing classifiers to accurately categorize feedback into various learner-centered components. The results demonstrated that the BERT-based classifiers outperformed other models in most feedback categories (achieving Cohen's κ up to 0.956 and F1 score up to 0.998), but showed relatively low performance in categories with less training data. This automated analysis aids in scrutinizing feedback quality, thereby supporting educators in enhancing their feedback practices to be more aligned with learner-centered principles.

Keywords

• Learner-centred feedback
• Automated feedback
• Learning analytics
• Machine learning
• Deep learning