AST+SVMNet: A Novel Decomposition Method for Micro-Expression Recognition Based on Fusion Attention and Improved Spatio- Temporal Convolution by Feature Transfer

Abstract

Micro-expression (ME) is spontaneous, rapid, and subtle facial mechanism that can reveal the concealed emotions. However, the short duration, low motion intensity, and small dataset of MEs make the extraction and learning of features from ME samples more challenging for existing micro-expression recognition (MER) methods. To address this issue, we propose a novel decomposition MER method, called AST+SVMNet, and the primary architecture of our method integrates improved fusion attention and spatio-temporal convolutional neural network, achieving efficient MER through feature transfer to SVM. This method consists of four main components: feature extraction, fusion attention, spatio-temporal feature extraction, and feature transfer modules. In the feature extraction part, we designed a novel ME texture feature called the image sequence difference feature (ISDF). It mitigates the negative impact of optical flow calculation noise on MER task when applying optical flow features simultaneously. In the fusion attention part, we designed a fusion attention module (FAM) that reduces the extraction of redundant information for ME samples, optimizing the extraction of finer-grained spatio-temporal information. In the third part, we reduced the parameter count through 2D and 3D Inception Modules without compromising the performance of spatio-temporal feature extraction. In the feature transfer part, we achieved rapid and efficient MER by training the SVM classifier through feature transfer on high-dimensional spatio-temporal features. Finally, the performance of our proposed method on four publicly available spontaneous ME datasets surpasses that of existing baseline methods in MER. In addition, through effectiveness experiments and ablation studies, we demonstrated the effectiveness of the proposed texture feature ISDF and the MER method AST+SVMNet.

Keywords

• Micro-expression recognition
• fusion attention
• spatio-temporal feature
• feature transfer
• differential feature