Automatic facial expression recognition combining texture and shape features from prominent facial regions

Abstract

Abstract Facial expression is one form of communication which being non‐verbal in nature precedes verbal communication in both origin and conception. Most of the existing methods for Automatic Facial Expression Recognition (AFER) are mainly focused on global feature extraction assuming that all facial regions contribute equal amount of discriminative information to predict the expression class. The detection and localization of facial regions that have significant contribution to expression recognition and extraction of highly discriminative feature distribution from those regions are not fully explored. The key contributions of the proposed work are developing novel feature distribution upon combining the discriminative power of shape and texture feature; determining the contribution of facial regions and identifying the prominent facial regions that hold abstract and highly discriminative information for expression recognition. The shape and texture features taken into consideration are Local Phase Quantization (LPQ), Local Binary Pattern (LBP), and Histogram of Oriented Gradients (HOG). Multiclass Support Vector Machine (MSVM) is used while one versus one classification. The proposed work is implemented on CK+, KDEF, and JAFFE benchmark facial expression datasets. The recognition rate of the proposed work is 94.2% on CK+ and 93.7% on KDEF, which is significantly more than the existing handcrafted feature‐based methods.

Keywords

• automatic facial expression recognition (AFER)
• facial local regions
• generalization capability
• high discriminative representation
• shape and texture feature fusion