MSFRNet: Two‐stream deep forgery detector via multi‐scale feature extraction

Abstract

Abstract Face forgery represented by DeepFake technique has raised severe societal concerns. Due to the different scales of tampering traces and the different resolutions of face images, adopting common processing pipelines and standard form of convolutional neural networks (CNNs) will lead to problems such as omission, redundancy, and bias when extracting key discriminative features. To solve the above issues, unlike most existing methods that treat face forensics as a vanilla binary classification task, the authors instead reformulate it as a multi‐scale object detection problem and propose a novel framework called MSFRNet based on multi‐scale feature extraction. Concretely, to alleviate the issues of features omission and redundancy, the authors construct a two‐stream prediction network, where the shallow branch discovers small‐scale objects such as tiny noise by capturing low‐level features with higher resolution and more details, while the deep stream exploits larger receptive fields to detect large‐scale blocky artefacts. Moreover, a multi‐scale feature extraction module is designed to enrich feature representations in each stream. To solve the problem of features bias and ensure that unbiased feature representations are learned, more appropriate data augmentation approaches are proposed by introducing counterfactual causal reasoning. Extensive experiments demonstrate that our framework outperforms most ordinary binary classifiers and achieves positive performance.

Keywords

• counterfactual causal reasoning
• DeepFake detection
• manipulation traces
• multi‐scale features