Machine learning's model-agnostic interpretability on the prediction of students' academic performance in video-conference-assisted online learning during the covid-19 pandemic

Abstract

Background: COVID-19 prompted a global shift to online learning, including video conference-assisted online learning (VCAOL), which necessitated educators understanding students' perspectives. Objective: This study aims to develop machine learning (ML) model-agnostic interpretability that could predict students' academic performance in VCAOL. Material and methods: Synthetic Minority Over-sampling Technique (SMOTE) and data augmentation were used to handle imbalanced data from small-scale datasets. The prediction model was developed using Random Forest (RF), Support Vector Machine (SVM), and Gaussian Naive Bayes (GNB). SHAP model-agnostic interpretability was used to interpret and comprehend prediction findings. The data was gathered from September 2022 to January 2023, resulting in 361 records. The research variables included students' academic performance as the dependent variable, and the video conference application (VC), learning material (LM), internet connection (IC), students' ability to learn (SL), and student knowledge (SK) as independent variables, which were mapped into 28 attributes. Result: The SMOTE improved the performance of three algorithms, with RF outperforming SVM and GNB in almost all tests, achieving an accuracy of 79.45%, precision of 75.71%, and recall of 79.45%. SHAP bar plots ranked attributes by importance demonstrated that “Performance,” “Frequency Constraint,” and “Increase Value” had a significant impact on prediction results. When we mapped the three attributes to our study perspective, we determined that SK and SL were the most important views for students to perform well in VCAOL. SHAP's beeswarm revealed students' performance in VCAOL was positively correlated with “Performance”, “Increase Value”, “Completing Project”, “Adequate Method”, “User Interface”, and “Feature”. As we mapped the three attributes to our study perspective, we found that SK, LM, SL, and VC were positively related to students' performance in VCAOL. Conclusion: The study highlighted the potential of ML in developing data-driven decision-making tools for predicting students' academic performance and identifying critical attributes in a prediction model.

Keywords

• Students' academic performance
• Video-conference-assisted online learning (VCAOL)
• Machine learning
• Model-agnostic