Facial Paralysis Symptom Detection Based on Facial Action Unit

Abstract

Facial paralysis refers to the abnormal behavior of facial muscles caused by a disorder of the facial nerve, mainly manifested as facial asymmetry. In recent years, deep learning has found extensive applications in facial paralysis detection research. However, most existing methods are constrained to assessing the severity of facial paralysis, thereby concealing crucial symptoms within black-box models. Compared to the severity of facial paralysis, the symptoms of facial paralysis are of greater significance to both physicians and patients. To address this issue, this paper proposes a facial paralysis symptom detection model based on facial action units (AUs). To enhance the accuracy of AU intensity prediction, a novel Difference Ensemble Method (DEM) is introduced. This method leverages differential information between frames within the same video to improve the accuracy of predictions for the current frame. Building upon the predicted AU intensity sequences for keyframes in a video, an interpretable model for detecting facial paralysis symptoms is designed. This model employs an active means to describe the asymmetry in facial muscle strength and utilizes co-occurrence matrices to detect synkinesis. It is noteworthy that DEM is exclusively trained on a dataset of normal faces but exhibits excellent performance when transferred to a facial paralysis dataset. Additionally, DEM exhibits higher accuracy in predicting AU intensity compared to existing methods. The F1 scores for detecting facial muscle function in the eyebrow, eye, and mouth regions with our proposed model are 80.0%, 79.23%, and 90.91%, respectively. To demonstrate the model’s performance, a synkinesis detection experiment is conducted, further validating its applicability in facial paralysis detection.

Keywords

• Co-occurrence matrix
• difference ensemble method
• deep learning
• facial paralysis symptom detection
• facial action unit