A Generalized Zero-Shot Deep Learning Classifier for Emotion Recognition Using Facial Expression Images

Abstract

In the real world environment, various micro and nano facial expressions are generated according to the mental state of thought, which leads to a change in an emotional state. A generalizable automatic emotion recognition system is needed to catch these micro and nano facial emotions correctly. Unfortunately, the literature lacks a generalizable Automatic Emotion Recognitions (AER) system. There are various AER systems based on learning and non-learning techniques, but they find it difficult to novel unseen facial expressions, which means they lack generalization ability. This study proposes a generalized deep learning classifier, Generalized Zero-Shot Convolutional Neural Network (GZS-ConvNet), to solve the problem of generalization. The objective of the proposed approach is to enhance the classifier’s generalizability by employing sophisticated adaptation methodologies. The newly developed classifier is meticulously calibrated and optimized using facial expression datasets. Six different datasets of facial expressions were employed to test the effectiveness of the proposed method: FER2013, AffectNet, RAF-DB, CK+, KDEF, and JAFEE. Zero-shot classification is performed on different facial expression datasets to justify the generalizability of the proposed model. The results prove that the proposed model is highly generalizable.

Keywords

• Automatic emotion recognition (AER)
• generalized zero shot (GZS)
• deep-learning classifier
• cross-domain generalization