Leveraging Inference: A Regression-Based Learner Performance Prediction System for Knowledge Tracing

Abstract

Learner modelling and performance prediction have seen numerous advances in the last decade which include Neural Network (NN) based approaches like Deep Knowledge Tracing (DKT), Factorisation machines for estimation and automatic detection of skill tags amongst others. Intelligent Tutoring Systems (ITSs), which try to tailor each user’s learning by adaptively scheduling problems depend upon learner modelling and the accurate prediction of learner performance to operate. ITSs have led to the availability of large-scale datasets enabling the development of richer models. The classical Bayesian approaches have been overtaken by faster, more scalable and more accurate models like DASH and DAS3H. Despite recent gains in prediction accuracy by NN based models, they are getting increasingly complex and computationally expensive. The issues with the scalability of recent models hinder their widespread adoption in Massive Online Open Courses (MOOCs) which outnumber ITSs by several magnitudes. NN models have very limited transferability. In this paper, the state-of-the-art approach involving Logistic Regression (LR) is examined on 8 public datasets and an alternative system based on variational encoding is presented. The proposed system outperforms the LR model in accuracy for 4 datasets. The new model is applied to the biggest Educational Data Mining (EDM) dataset available and shows that training on such a large dataset is possible without batching, which is not possible by other techniques. The paper focuses on the increase in scalability with a smaller memory footprint (requiring 22.5 ± 7.45 % less memory). The system also had a reduction in runtime for 5 among the 8 datasets. However, the variance in reduction is high with an average of $10.12\pm 21.55 \%$ reduced run times. The proposed system was developed specifically for lightweight MOOCs with limited Learning Management Systems (LMS) such as Moodle. By estimating a learner ability parameter, the proposed system provides richer information to downstream applications like adaptive scheduling. The proposed model retains a high level of interpretability and transferability.

Keywords

• Intelligent tutoring systems
• performance prediction
• regression models
• stochastic variational inference
• learner modeling