Mining label-free consistency regularization for noisy facial expression recognition

Abstract

Abstract Noisy labels are unavoidable in facial expression recognition (FER) task, significantly hindering FER performance in real-world scenarios. Recent advances tackle this problem by leveraging uncertainty for sample partitioning or constructing label distributions. However, these approaches primarily depend on labels, leading to confirmation bias issues and performance degradation. We argue that mining both label-independent features and label-dependent information can mitigate the confirmation bias induced by noisy labels. In this paper, we propose MCR, that is, mining simple yet effective label-free consistency regularization (MCR) to learn robust representations against noisy labels. The proposed MCR incorporates three label-free consistency regularizations: instance-level embedding consistency regularization, pairwise distance consistency regularization, and neighbour consistency regularization. Initially, we employ instance-level embedding consistency regularization to learn instance-level discriminative information from identical facial samples under perturbations in an unsupervised manner. This facilitates the efficacy of mitigating inherent noise in data. Subsequently, a pairwise distance consistency regularization is constructed to regularize the classifier and alleviate bias induced by noisy labels. Finally, we use the neighbour consistency regularization to further strengthen the discriminative capability of the model against noise. Benefiting from the advantages of these three label-free consistency regularizations, MCR can learn discriminative and robust representations against noise. Extensive experimental results demonstrate the superior performance of MCR on three popular in-the-wild facial expression datasets, including RAF-DB, FERPlus, and AffectNet. Moreover, MCR demonstrates superior generalization capability on other datasets with noisy labels, such as CIFAR100 and Tiny-ImageNet.

Keywords

• Facial expression recognition
• Label-free consistency regularization
• Unsupervised contrastive learning
• Noisy label learning