Deepfake Detection Algorithm Based on Dual-Branch Data Augmentation and Modified Attention Mechanism

Abstract

Mainstream deepfake detection algorithms generally fail to fully extract forgery traces and have low accuracy when detecting forged images with natural corruptions or human damage. On this basis, a new algorithm based on an adversarial dual-branch data augmentation framework and a modified attention mechanism is proposed in this paper to improve the robustness of detection models. First, this paper combines the traditional random sampling augmentation method with the adversarial sample idea to enhance and expand the forged images in data preprocessing. Then, we obtain training samples with diversity and hardness uniformity. Meanwhile, a new attention mechanism is modified and added to the ResNet50 model. The improved model serves as the backbone, effectively increasing the weight of forged traces in the multi-scale feature maps. The Jensen–Shannon divergence loss and cosine annealing algorithms are introduced into the training process to improve the model’s accuracy and convergence speed. The proposed algorithm is validated on standard and corrupted datasets. The experiments show that the algorithm proposed in this paper significantly improves effectiveness and robustness, with accuracies of 4.16%, 7.37%, and 3.87% better than the baseline model on DeepFakes, FaceSwap, and FaceShifer, respectively. Most importantly, its detection performance on the corrupted datasets DeepFakes-C, FaceSwap-C, and FaceShifer-C is much higher than that of mainstream methods.

Keywords

• deepfake detection
• data augmentation
• attention mechanism
• robustness