Deep Convolutional Neural Network Algorithm Fusing Global and Local Features

Abstract

In order to further improve the accuracy of facial expression recognition, a deep convolutional neural network algorithm fusing global and local features (GL-DCNN) is proposed. The algorithm consists of two improved convolutional neural network branches, global branch and local branch, which are used to extract global features and local features respectively. The features of the two branches are weighted and fused, and the fused features are used for classification. Firstly, global features are extracted. The global branch is based on transfer learning, and the improved VGG19 network model is used for feature extraction. Secondly, local features are extracted. In the local branch, central symmetric local binary pattern (CSLBP) algorithm is used for the first feature extraction, and the local texture information of the original image is obtained, which is input into shallow convolutional neural network for the second feature extraction, so that the local features related to facial expressions are automatically extracted. Thirdly, two cascaded fully connected layers are used to reduce the dimension of the features of the two branches, and different weights are assigned to them for weighted fusion. Finally, softmax classifier is used for classification. The experiment is validated on CK+ and JAFFE datasets, and the classification accuracy is over 95% and 93%, respectively. Compared with other five algorithms, this algorithm has a good overall performance, good recognition effect and good robustness, which can provide an effective basis for facial expression recognition.

Keywords

• |facial expression recognition|feature fusion|convolutional neural networks (cnn)|deep learning