Facial Expression Recognition to Detect Student Engagement in Online Lectures

Abstract

In synchronous online lectures, the lecturers often provide the lecture material directly through video conference technology. On the other hand, there are many students who do not pay attention to the lecturers when they are participating in online lectures. As a consequence, in this research, an application was developed to assist lecturers in gathering data regarding the degree to which students who participate in online lectures pay attention to the presented information. The application employed a convolutional neural network (CNN) model to recognize each student's facial expressions and place them into one of two classes: either engaged or disengaged. The captured student facial image was preprocessed to facilitate the classification process. The preprocessing stage consisted of image conversion to gray scale, face detection using the Haar-Cascade Classifier model, and a median filter to reduce noise. In the process of designing a CNN model, three different hyperparameter tuning scenarios were implemented. These tuning scenarios aimed to obtain the best possible CNN model by determining which CNN model hyperparameters were the most optimal. The results of the experiments indicate that the CNN model from the second scenario has the highest level of accuracy in terms of recognizing facial expressions, coming in at 86%. The results of this research have been tested to measure the level of student participation in online lectures. The trial results show that the proposed application can help lecturers evaluate student engagement during online lectures.

Keywords

• facial expression recognition
• convolutional neural network
• student engagement
• online lectures