Facial Expression Recognition in the Wild Using Face Graph and Attention

Abstract

Facial expression recognition (FER) in the wild from various viewpoints, lighting conditions, face poses, scales, and occlusions is an extremely challenging field of research. In this study, we construct a face graph by selecting action units that play an important role in changing facial expressions, and we propose an algorithm for recognizing facial expressions using a graph convolutional network (GCN). We first generated an attention map that can highlight action units to extract important facial expression features from faces in the wild. After feature extraction, a face graph is constructed by combining the attention map with face patches, and changes in expression in the wild are recognized using a GCN. Through comparative experiments conducted using both lab-controlled and wild datasets, we prove that the proposed method is the most suitable FER approach for use with image datasets captured in the wild and those under well-controlled indoor conditions.

Keywords

• Facial expression recognition
• action unit
• attention map
• face graph
• graph convolutional network