Predicting the Reader’s English Level From Reading Fixation Patterns Using the Siamese Convolutional Neural Network

Abstract

Among numerous functions performed by the human eye, reading is a common task that best reflects an individual’s understanding and cognitive patterns. Previous studies showed that text comprehension may be determined by comprehension monitoring, a metacognitive process that evaluates and regulates the pattern of comprehension. Herein, we propose a hypothesis: an individual’s cognitive pattern during reading is predictive of the level of reading comprehension. According to the criteria of the College English Test Band Six (CET-6), 80 participants (sophomore and junior) were divided into a pass group (n = 40) and a non-pass group (n = 40). Heatmaps of eye fixation counts were collected by an eye-tracker while each participant executed four reading comprehension tests. Using these heatmaps as inputs, we proposed the Siamese convolutional neural network models to predict the English level of participants. Both strategies of “Trained from scratch” and “Pretrained with fine-tuning” were employed. “Soft Voting” was applied to integrate the predictions from four tests. Results showed that the Siamese network model trained by the datasets with the cluster radius of fixation equal to 25 pixels and connection layer by L1 norm distance has a satisfactory or superior performance to other comparative experiments. The AUC values of Siamese networks trained by the two strategies reach 0.941 and 0.956, respectively. This indicates that the individual reading cognitive pattern captured by the eye-tracker can predict the level of reading comprehension through advanced deep learning models.

Keywords

• Comprehension monitoring
• eye-tracker
• reading comprehension
• Siamese network
• transfer learning