Micro-Expression Recognition Based on Spatio–Temporal Capsule Network

Abstract

Micro-expression (ME) is a subtle and spontaneous natural facial mechanism that is one of the essential psychological stress responses. Due to its accuracy and uncontrollability for psychological expressions, ME has crucial applications in many psychologically related fields. However, due to small data volume and high data redundancy, existing micro-expression recognition (MER) methods cannot balance accuracy and recognition speed. Therefore, we propose a deep learning algorithm based on a spatio–temporal capsule network (STCP-Net). It reduces recognition time while ensuring the accuracy rate. STCP-Net consists of four parts: a jitter removal module, a differential feature extraction module, a spatio–temporal capsule (STCP) module, and a fully connected layer. The first two modules extract diversifying differential features more accurately and reduce the impact of head jitter. The STCP module extracts spatio–temporal features progressively layer by layer, fully considering the temporal and spatial relationship between features. Finally, this study conducts sufficient experiments based on the Leave One Subject Out (LOSO) cross-validation protocol with standard datasets. The final results and analysis show that the algorithm is advanced and effective.

Keywords

• Capsule network
• differential features
• deep learning
• micro-expression recognition
• spatio-temporal features