Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments

Abstract

In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. We present a machine learning (ML) model which identifies the phase of cognitive presence exhibited by a student’s post and suggest future applications of such a model to help online students develop higher-order thinking. We collect discussion forum transcript data from two online courses: CS1301 (an introductory computer programming MOOC) offered by edX and CS6601 (a graduate course on artificial intelligence) which uses the Piazza online discussion tool. We manually code a random sample of students’ posts based on the Community of Inquiry coding scheme and explore trends in cognitive presence within and across the courses. We further use this coded data to analyze the relationship between students’ observed cognitive presence and course grades. In terms of testing and building an ML model, we use a Bidirectional Encoder Representations from Transformers model that uses a deep learning technique to train large text corpus and fine-tune the language model. Our results suggest that deeper cognitive engagement with course concepts, as expressed by higher cognitive presence, are associated with better learning outcomes for students in both course settings. Our ML approach achieves 92.5% accuracy on the classification task, motivating the use of ML for instructional interventions in online courses. We expect that our research study will not only contribute to extending the literature on cognitive presence but also have a beneficial impact on online instructors or curriculum developers in higher education.

Keywords

• cognitive presence
• discussion forums
• machine learning
• higher education