Graph Convolutional Neural Networks for Micro-Expression Recognition—Fusion of Facial Action Units for Optical Flow Extraction

Abstract

Micro-expression recognition is an important problem in the field of computer vision and affective computing. To improve the accuracy of micro-expression recognition, the study proposes a novel graph convolutional neural network model oriented to micro-expression recognition, which divides facial features into regions and uses the optical flow method for feature extraction of facial action units. The model utilizes graph structure to encode facial features intuitively, while obtaining dynamic changes of facial features through the change information of optical flow, thus obtaining richer micro-expression feature information. The results show that the model performance can be maximized when the model is set to 5-way 5-shot and is taken as 1.4, at which time the model’s accuracy on the dataset CAMSE II is 79.168%. The proposed algorithm performs well when compared with other algorithms in terms of accuracy and F1 score, the proposed algorithm realizes an accuracy of 0.795 on the CAMSE II dataset compared to the other algorithms which is up to 0.785. The accuracy of the proposed algorithm on the SAMM dataset is 0.738, which is only lower than that of the spatio-temporal recurrent convolutional neural network. The algorithm proposed in the study shows good performance in micro-expression recognition and promotes the development of the field of computer vision and affective computing.

Keywords

• Facial action
• feature extraction
• graph convolutional neural network
• micro-expression recognition