FFDL: Feature Fusion-Based Deep Learning Method Utilizing Federated Learning for Forged Face Detection

Abstract

The widespread adoption of advanced technologies may be responsible for the extensive dissemination of forged photographs and videos on the Internet. This could potentially result in the proliferation of fraudulent identities online, raising safety concerns in society. The traditional method for detecting forgery, commonly referred to as the classical forgery method, lacks the capability to accurately identify such fraudulent activities. This limitation arises because these algorithms are trained on publicly available centralized datasets and do not prioritize privacy and security considerations. Consequently, they adversely affect the ability to detect counterfeit content. As a potential solution to this problem, we employed a highly effective deep learning methodology rooted in federated learning. We introduced a novel deep learning approach that combines features to assess the authenticity of photographs and videos shared on social media platforms. The proposed model was trained using three widely recognized forensic datasets: FaceForensics++, Deepforensic-1.0, and WildDeepfake. Visual features were extracted using two widely recognized deep learning approaches, namelyInception and Xception. These features were then combined into a feature vector using Canonical Correlation Analysis, and Convolutional Neural Networks were trained on these features to identify manipulated images and videos. The experiments were carried out with publicly available datasets and involved changing several parameters. Finally, the proposed model’s performance was compared with other deep learning models within federated learning environments to identify forgeries. Our proposed approach demonstrated exceptional performance, achieving an accuracy rate of 98.99% when evaluated on the merged dataset.

Keywords

• Fake detection
• deep learning
• feature fusion
• federated learning
• privacy-preserving