IEEE Access (Jan 2023)
Model Focus Improves Performance of Deep Learning-Based Synthetic Face Detectors
Abstract
Deep learning-based models generalize better to unknown data samples after being guided “where to look” by incorporating human perception into training strategies. We made an observation that the entropy of the model’s salience trained in that way is lower when compared to salience entropy computed for models training without human perceptual intelligence. The research problem addressed by this paper is whether lowering the entropy of model’s class activation map helps in further increasing the performance, on top of the performance increase we observe for human saliency-based model’s training. In this paper we propose and evaluate four new entropy-based loss functions controlling the model’s focus, covering the full range of the level of such control, from none to its “aggresive” minimization. We show, using a problem of synthetic face detection, that improving the model’s focus, through lowering entropy by the proposed loss components, leads to models that perform better in an open-set scenario (in which the test samples are synthesized by unknown generative models): the obtained average Area Under the ROC curve (AUROC) ranges from 0.72 to 0.78, compared to AUROC = 0.64 observed for a state-of-the-art human-salience-only-based control of the model’s focus. We also show that optimal performance is obtained when the model’s loss function blends three aspects: regular classification performance, low-entropy of the model’s focus, and closeness of the model’s focus to human saliency. The major conclusion from this work is that maximization of the model’s focus is an important regularizer allowing the models to generalize better in an open set scenario. Future work directions include methods of blending classification-, human salience-, and model’s salience entropy-based loss components to achieve optimal performance in other domains than the synthetic face detection.
Keywords
