Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection

Abstract

Abstract Due to the increasing sophistication of face forgery techniques, the images generated are becoming more and more realistic and difficult for human eyes to distinguish. These face forgery techniques can cause problems such as fraud and social engineering attacks in facial recognition and identity verification areas. Therefore, researchers have worked on face forgery detection studies and have made significant progress. Current face forgery detection algorithms achieve high detection accuracy within-dataset. However, it is difficult to achieve satisfactory generalization performance in cross-dataset scenarios. In order to improve the cross-dataset detection performance of the model, this paper proposes a multi-feature fusion network based on two-stream extraction and multi-scale enhancement. First, we design a two-stream feature extraction module to obtain richer feature information. Secondly, the multi-scale feature enhancement module is proposed to focus the model more on information related to the current sub-region from different scales. Finally, the forgery detection module calculates the overlap between the features of the input image and real images during the training phase to determine the forgery regions. The method encourages the model to mine forgery features and learns generic and robust features not limited to a particular feature. Thus, the model achieves high detection accuracy and performance. We achieve the AUC of 99.70% and 90.71% on FaceForensics++ and WildDeepfake datasets. The generalization experiments on Celeb-DF-v2 and WildDeepfake datasets achieve the AUC of 80.16% and 65.15%. Comparison experiments with multiple methods on other benchmark datasets confirm the superior generalization performance of our proposed method while ensuring model detection accuracy. Our code can be found at: https://github.com/1241128239/MFNet .

Keywords

• Deepfake detection
• Face forgery detection
• Deep learning
• Attention mechanism
• Feature extraction
• Feature enhancement