Graph Reasoning-Based Emotion Recognition Network

Abstract

Semantic information from images can be used to improve the performance of deep learning methods in recognizing human emotions. In this paper, we propose a novel framework based on the graph convolutional network for emotion recognition by utilizing the semantic relationships of different regions. First, we extract the salient image regions within video frame clips by using the bottom-up attention module to construct the node features of a graph. Then, we build the graphs containing the node features and the semantic correlations of nodes by using the graph convolutional network. For refinement, each node feature of graph vectors is enhanced via a gated recurrent unit consisting of gate and memory units to remove redundant feature information. Experimental results show that our proposed method achieves superior performance over state-of-the-art approaches for the emotion recognition on the CEAR and AFEW datasets.

Keywords

• Emotion recognition
• graph convolutional neural networks
• contextual spatiotemporal features