Facial Expression Recognition Using Hierarchical Features With Three-Channel Convolutional Neural Network

Abstract

Aiming at the problem of insufficient feature extraction and low recognition rate of traditional convolutional neural network in facial expression recognition, a multi-layer feature recognition algorithm based on three-channel convolutional neural network (HFT-CNN) was proposed. In the three-channel convolutional neural network, one channel trained the local image of the eyes and eyebrows extracted from the facial expression image, another channel trained the local image of the mouth extracted from the facial expression image, and the last channel tained the whole facial expression image. In order to better extract the facial features of the three channel, the higher-order tensor singular value decomposition was used to decompose the convolution kernel to remove the redundancy.The detail convolution kernel was constructed for the eyes and eyebrow and mouth image channeles, while the contour convolution kernel was constructed for the whole facial expression image channel. Finally, the features extracted from the three network channels were fused to obtain the expression classification results. Facial expression recognition experiments were carried out on CK+ and JAFFE facial expression datasets respectively, and HFT-CNN was compared to traditional deep neural networks and the state of the art deep neural networks, the network structure in this paper achieved higher recognition rates with fewer network layers.

Keywords

• Facial expression recognition
• convolutional neural network
• convolutional kernel weight
• three-channel
• multi-channel fusion