Where Did I Get Lost? A Prototype Tool for Predicting Local-Level Comprehension Difficulties With Wearables and Explainable Machine Learning

Abstract

In this study, we present iMind, a novel prototype tool designed to predict individuals’ comprehension difficulties with online content at local levels, such as paragraphs or multiple lines. iMind seamlessly captures the physiological responses of the Autonomic Nervous System (ANS) using commercially available wearable sensors (wristbands) in conjunction with a desktop eye-tracker to identify potentially difficult content sections. The machine learning model of iMind successfully predicts comprehension difficulties in English content at local levels, achieving a mean accuracy of 0.70 (±0.02), a precision mean of 0.70 (±0.03), a mean recall of 0.75 (±0.03), and an F1 score mean of 0.72 (±0.02). Furthermore, this document introduces a comprehensive risk analysis model that not only explains the predictions of iMind but also quantifies the risk scores associated with specific sections of content deemed difficult to comprehend. This integration enhances the interpretability of iMind’s outcomes, providing clearer insights into which content sections may require further simplification or support for improved comprehension.

Keywords

• Content comprehension
• electordermal activity
• eye-tracking
• machine learning
• pulse rate variability
• wearable sensors