Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism

Abstract

Micro-Expression Recognition is crucial in various fields such as behavioral analysis, security, and psychological studies, offering valuable insights into subtle and often concealed emotional states. Despite significant advancements in deep learning models, challenges persist in accurately handling the nuanced and fleeting nature of micro-expressions, particularly when applied across diverse datasets with varied expressions. Existing models often struggle with precision and adaptability, leading to inconsistent recognition performance. To address these limitations, we propose the Convolutional Variational Attention Transformer (ConVAT), a novel model that leverages a multi-head attention mechanism integrated with convolutional networks, optimized specifically for detailed micro-expression analysis. Our methodology employs the Leave-One-Subject-Out (LOSO) cross-validation technique across three widely used datasets: SAMM, CASME II, and SMIC. The results demonstrate the effectiveness of ConVAT, achieving impressive performance with 98.73% accuracy on the SAMM dataset, 97.95% on the SMIC dataset, and 97.65% on CASME II. These outcomes not only surpass current state-of-the-art benchmarks but also highlight ConVAT’s robustness and reliability in capturing micro-expressions, marking a significant advancement toward developing sophisticated automated systems for real-world applications in micro-expression recognition.

Keywords

• ConVAT
• convolutional neural networks
• LOSO cross-validation
• micro-expression recognition
• multi-head attention