Brain Region-Based Vigilance Assessment Using Electroencephalography and Eye Tracking Data Fusion

Abstract

Vigilance is the capacity to remain alert for an extended time while performing a task. Staying alert is obligatory in many jobs, particularly those that involve monitoring, such as surveillance tasks, security monitoring, and air traffic control. These monitoring tasks require a specific level of arousal to maintain an adequate level of cognitive efficiency. In this study, we investigate the possibility of assessing the vigilance levels using a fusion of electroencephalography (EEG) and eye tracking data. Vigilance levels were established by performing a modified version of the Stroop color word task (SCWT) for 30 minutes. Feature-level fusion based on the canonical correlation analysis (CCA) was employed to each brain region to improve the classification accuracy of vigilance level assessment. Results obtained using support vector machines (SVM) classifier show that fusion of EEG+eye tracking modalities has improved the classification accuracy compared to individual modality. The EEG+Eye tracking fusion on the right central brain region achieved the highest classification accuracy of 97.4 ± 1.3%, compared to the individual Beta EEG with 92.0±7.3% and Eye tracking with 76.8±8.4%, respectively. Likewise, EEG and Eye tracking fusion on the right frontal region showed classification accuracy of 96.9 ± 1.1% for both the Alpha and Beta bands. Meanwhile, when all brain regions were utilized, the highest classification accuracy of EEG+Eye tracking was 96.8 ± 0.6% using Delta band compared to the EEG alone with 88.18 ± 8.5% and eye tracking alone with 76.8 ± 8.4 %, respectively. The overall results showed that vigilance is a brain region specific and the fusion of EEG+ and Eye tracking data using CCA has significantly improved the classification accuracy of vigilance levels assessment.

Keywords

• Data fusion
• EEG
• eye tracking
• vigilance assessment
• canonical correlation analysis (CCA)
• machine learning