A Deep Learning Model Based On Multi-granularity Facial Features And LSTM Network For Driver Drowsiness Detection

Abstract

Driver drowsiness can cause serious harm to drivers and other road participants. Exploring objective and efficient methods for detecting driver drowsiness has important application value for ensuring road safety. Considering the information complementary between local and global facial features for drowsiness detection, as well as the advantages of deep learning models in information mining, this paper proposes a deep learning model based on multi-granularity facial features and Long Short Term Memory (LSTM) network for driver drowsiness detection. To obtain local facial feature information, face detection and facial landmarks location are implemented based on Practical Facial Landmark Detector (PFLD). The local representation features of the eyes and mouth, as well as the head pose feature, are calculated from the coordinate information of facial landmarks. Furthermore, a global representation learning Vision Transformer (ViT) model that trained on the NTHU-DDD dataset to obtain higher-level semantic information. Due to drowsiness has an accumulative property, an LSTM network that takes the local and global multi-granularity representation features as input to further mine the drowsy clues in the temporal dimension. A large number of comparative experiments are conducted on the public NTHU-DDD dataset, and the results show that the proposed method outperformed other methods, achieving a detection accuracy of 93.15%. Experimental results show that the method can achieve much higher accuracy and can provide an alternative solution for the driver assistance system.

Keywords

• driver drowsiness detection
• deep learning model
• multi-granularity representation features
• vision transformer
• long short term memory network