Decoupling Forgery Semantics for Generalizable Deepfake Detection

• URL: http://arxiv.org/abs/2406.09739v2
• Date: Mon, 19 Aug 2024 06:27:49 GMT
• Title: Decoupling Forgery Semantics for Generalizable Deepfake Detection
• Authors: Wei Ye, Xinan He, Feng Ding,
• Abstract summary: We propose a novel method for detecting DeepFakes, enhancing the generalization of detection through semantic decoupling. Evaluation on FF++, Celeb-DF, DFD, and DFDC datasets showcases our method's excellent detection and generalization performance.
• Score: 6.1822981823804835
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a novel method for detecting DeepFakes, enhancing the generalization of detection through semantic decoupling. There are now multiple DeepFake forgery technologies that not only possess unique forgery semantics but may also share common forgery semantics. The unique forgery semantics and irrelevant content semantics may promote over-fitting and hamper generalization for DeepFake detectors. For our proposed method, after decoupling, the common forgery semantics could be extracted from DeepFakes, and subsequently be employed for developing the generalizability of DeepFake detectors. Also, to pursue additional generalizability, we designed an adaptive high-pass module and a two-stage training strategy to improve the independence of decoupled semantics. Evaluation on FF++, Celeb-DF, DFD, and DFDC datasets showcases our method's excellent detection and generalization performance. Code is available at: https://github.com/leaffeall/DFS-GDD.

Related papers

• Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection [43.2796409299818]
  Deepfakes are becoming a nuisance to law enforcement authorities and the general public.<n>Existing deepfake detectors are struggling to keep up with the pace of improvements in deepfake generation.<n>This paper proposes a new strategy that leverages coarse-to-fine spatial information, semantic information, and their interactions.
  arXiv  Detail & Related papers  (2025-05-08T01:49:53Z)
• Securing Social Media Against Deepfakes using Identity, Behavioral, and Geometric Signatures [6.3947036687002985]
  Trust in social media is a growing concern due to its ability to influence significant societal changes. Deepfake multimedia undermine the authenticity of shared content. Existing detection techniques tend to perform well only on specific types of deepfakes they were trained on.
  arXiv  Detail & Related papers  (2024-12-07T01:17:21Z)
• Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection [15.857961926916465]
  We present a novel general deepfake detection method, called textbfCurricular textbfDynamic textbfForgery textbfAugmentation (CDFA) CDFA jointly trains a deepfake detector with a forgery augmentation policy network. We show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors.
  arXiv  Detail & Related papers  (2024-09-22T13:51:22Z)
• Semantics-Oriented Multitask Learning for DeepFake Detection: A Joint Embedding Approach [77.65459419417533]
  We propose an automatic dataset expansion technique to support semantics-oriented DeepFake detection tasks. We also resort to joint embedding of face images and their corresponding labels for prediction. Our method improves the generalizability of DeepFake detection and renders some degree of model interpretation by providing human-understandable explanations.
  arXiv  Detail & Related papers  (2024-08-29T07:11:50Z)
• Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection [33.20064862916194]
  This paper aims to learn potential cross-modal correlation to enhance deepfake detection towards various generation scenarios. Our approach introduces a correlation distillation task, which models the inherent cross-modal correlation based on content information. We present the Cross-Modal Deepfake dataset with four generation methods to evaluate the detection of diverse cross-modal deepfakes.
  arXiv  Detail & Related papers  (2024-04-30T00:25:44Z)
• Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection [57.646582245834324]
  We propose a simple yet effective deepfake detector called LSDA. It is based on a idea: representations with a wider variety of forgeries should be able to learn a more generalizable decision boundary. We show that our proposed method is surprisingly effective and transcends state-of-the-art detectors across several widely used benchmarks.
  arXiv  Detail & Related papers  (2023-11-19T09:41:10Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection [73.66077273888018]
  Existing deepfake detection methods fail to generalize well to unseen or degraded samples. High-level semantics are indispensable recipes for generalizable forgery detection. DeepFake-Adapter is first parameter-efficient tuning approach for deepfake detection.
  arXiv  Detail & Related papers  (2023-06-01T16:23:22Z)
• Learning Pairwise Interaction for Generalizable DeepFake Detection [20.723277551489186]
  A fast-paced development of DeepFake generation techniques challenge the detection schemes designed for known type DeepFakes. We propose a new approach, Multi-Channel Xception Attention Pairwise Interaction (MCX-API), that exploits the power of pairwise learning and complementary information from different color space representations. Our experiments indicate that our proposed method can generalize better than the state-of-the-art Deepfakes detectors.
  arXiv  Detail & Related papers  (2023-02-26T10:39:08Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.