Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection

• URL: http://arxiv.org/abs/2409.14444v1
• Date: Sun, 22 Sep 2024 13:51:22 GMT
• Title: Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection
• Authors: Yuzhen Lin, Wentang Song, Bin Li, Yuezun Li, Jiangqun Ni, Han Chen, Qiushi Li,
• Abstract summary: We present a novel general deepfake detection method, called textbfCurricular textbfDynamic textbfForgery textbfAugmentation (CDFA) CDFA jointly trains a deepfake detector with a forgery augmentation policy network. We show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors.
• Score: 15.857961926916465
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Previous studies in deepfake detection have shown promising results when testing face forgeries from the same dataset as the training. However, the problem remains challenging when one tries to generalize the detector to forgeries from unseen datasets and created by unseen methods. In this work, we present a novel general deepfake detection method, called \textbf{C}urricular \textbf{D}ynamic \textbf{F}orgery \textbf{A}ugmentation (CDFA), which jointly trains a deepfake detector with a forgery augmentation policy network. Unlike the previous works, we propose to progressively apply forgery augmentations following a monotonic curriculum during the training. We further propose a dynamic forgery searching strategy to select one suitable forgery augmentation operation for each image varying between training stages, producing a forgery augmentation policy optimized for better generalization. In addition, we propose a novel forgery augmentation named self-shifted blending image to simply imitate the temporal inconsistency of deepfake generation. Comprehensive experiments show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors in a plug-and-play way, and make them attain superior performances over the existing methods in several benchmark datasets.

Related papers

• Towards General Deepfake Detection with Dynamic Curriculum [4.622705420257596]
  We propose to introduce the sample hardness into the training of deepfake detectors via the curriculum learning paradigm. We present a novel simple yet effective strategy, named Dynamic Facial Forensic Curriculum (DFFC), which makes the model gradually focus on hard samples during the training. Comprehensive experiments show that DFFC can improve both within- and cross-dataset performance of various kinds of end-to-end deepfake detectors.
  arXiv  Detail & Related papers  (2024-10-15T00:58:09Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
  We introduce an approach that is both general and parameter-efficient for face forgery detection. We design a forgery-style mixture formulation that augments the diversity of forgery source domains. We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection [96.539862328788]
  Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF) Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-21T08:58:49Z)
• SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes [7.553507857251396]
  We propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task. SeeABLE pushes perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss. We show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities.
  arXiv  Detail & Related papers  (2022-11-21T09:38:30Z)
• On Improving Cross-dataset Generalization of Deepfake Detectors [1.0152838128195467]
  Facial manipulation by deep fake has caused major security risks and raised severe societal concerns. We formulate deep fake detection as a hybrid combination of supervised and reinforcement learning (RL) to improve its cross-dataset generalization performance. We demonstrate the superiority of our method over existing published research in cross-dataset generalization of deep fake detectors, thus obtaining state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-08T20:34:53Z)
• Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection [41.27496491339225]
  This work addresses the generalizable deepfake detection from a simple principle. We propose to enrich the "diversity" of forgeries by synthesizing augmented forgeries with a pool of forgery configurations. We also propose to use the adversarial training strategy to dynamically synthesize the most challenging forgeries to the current model.
  arXiv  Detail & Related papers  (2022-03-23T05:52:23Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.