Enhanced Driver Drowsiness Detection Model Using Multi-Level Features Fusion and a Long-Short-Term Recurrent Neural Network

Abstract

Drowsiness driving poses a significant risk to road safety, necessitating effective drowsiness detection models. Most of the prior research has primarily relied on composite facial-based features, mainly focusing on the mouth and/or eye states, to identify drowsiness status. However, these models tend to overlook crucial information from input signals, resulting in suboptimal detection accuracy. Moreover, the absence of suitable algorithms and techniques for extracting other essential facial features, such as the eyebrow and nostril, further impacts the accuracy of drowsiness detection. To address these limitations, this study introduces an innovative algorithm and a technique for extracting drowsiness-related information from the eyebrow and nostril regions. Additionally, we propose a method, leveraging four composite facial-based drowsiness features; eyebrow, nostril, eye, and mouth states as inputs to a Convolutional Neural Network (CNN). A novel multilevel feature fusion method is employed to effectively combine the deep representations of these drowsiness-related features. The final step involves employing a Long-short-term memory (LSTM) recurrent neural network to classify the drowsiness status of drivers. Our proposed model is rigorously evaluated using the National Tsing Hua University drowsy driver detection (NTHU-DDD) video dataset. The experimental results demonstrate an impressive accuracy in different scenarios, and the accuracy result reached 0.973, showcasing the effectiveness of our approach in enhancing drowsiness detection accuracy and promoting road safety.

Keywords

• driver drowsiness detection
• facial-based features
• multilevel feature fusion
• recurrent neural network
• road safety