A Quality-Centric Framework for Generic Deepfake Detection

• URL: http://arxiv.org/abs/2411.05335v3
• Date: Sat, 17 May 2025 10:04:20 GMT
• Title: A Quality-Centric Framework for Generic Deepfake Detection
• Authors: Wentang Song, Zhiyuan Yan, Yuzhen Lin, Taiping Yao, Changsheng Chen, Shen Chen, Yandan Zhao, Shouhong Ding, Bin Li,
• Abstract summary: We propose a novel quality-centric framework for generic deepfake detection.<n>It is composed of a Quality Evaluator, a low-quality data enhancement module, and a learning pacing strategy.<n>Our framework is inspired by curriculum learning, which is designed to gradually enable the detector to learn more challenging deepfake samples.
• Score: 41.1730636042585
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Detecting AI-generated images, particularly deepfakes, has become increasingly crucial, with the primary challenge being the generalization to previously unseen manipulation methods. This paper tackles this issue by leveraging the forgery quality of training data to improve the generalization performance of existing deepfake detectors. Generally, the forgery quality of different deepfakes varies: some have easily recognizable forgery clues, while others are highly realistic. Existing works often train detectors on a mix of deepfakes with varying forgery qualities, potentially leading detectors to short-cut the easy-to-spot artifacts from low-quality forgery samples, thereby hurting generalization performance. To tackle this issue, we propose a novel quality-centric framework for generic deepfake detection, which is composed of a Quality Evaluator, a low-quality data enhancement module, and a learning pacing strategy that explicitly incorporates forgery quality into the training process. Our framework is inspired by curriculum learning, which is designed to gradually enable the detector to learn more challenging deepfake samples, starting with easier samples and progressing to more realistic ones. We employ both static and dynamic assessments to assess the forgery quality, combining their scores to produce a final rating for each training sample. The rating score guides the selection of deepfake samples for training, with higher-rated samples having a higher probability of being chosen. Furthermore, we propose a novel frequency data augmentation method specifically designed for low-quality forgery samples, which helps to reduce obvious forgery traces and improve their overall realism. Extensive experiments demonstrate that our proposed framework can be applied plug-and-play to existing detection models and significantly enhance their generalization performance in detection.

Related papers

• Few-Shot Learning with Adaptive Weight Masking in Conditional GANs [2.4299671488193497]
  This paper introduces a novel approach to few-shot learning by employing a Residual Weight Masking Conditional Generative Adversarial Network (RWM-CGAN) for data augmentation. The proposed model integrates residual units within the generator to enhance network depth and sample quality, coupled with a weight mask regularization technique in the discriminator to improve feature learning from small-sample categories.
  arXiv  Detail & Related papers  (2024-12-04T08:10:48Z)
• Towards General Deepfake Detection with Dynamic Curriculum [4.622705420257596]
  We propose to introduce the sample hardness into the training of deepfake detectors via the curriculum learning paradigm. We present a novel simple yet effective strategy, named Dynamic Facial Forensic Curriculum (DFFC), which makes the model gradually focus on hard samples during the training. Comprehensive experiments show that DFFC can improve both within- and cross-dataset performance of various kinds of end-to-end deepfake detectors.
  arXiv  Detail & Related papers  (2024-10-15T00:58:09Z)
• Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection [15.857961926916465]
  We present a novel general deepfake detection method, called textbfCurricular textbfDynamic textbfForgery textbfAugmentation (CDFA) CDFA jointly trains a deepfake detector with a forgery augmentation policy network. We show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors.
  arXiv  Detail & Related papers  (2024-09-22T13:51:22Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models [52.04189118767758]
  Generalization is a main issue for current audio deepfake detectors. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection.
  arXiv  Detail & Related papers  (2024-05-03T15:27:11Z)
• Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learning [26.517887637150594]
  Deepfake has recently raised a plethora of societal concerns over its possible security threats and dissemination of fake information. Most SOTA approaches are limited by using a single specific model for detecting certain deepfake video quality type. We propose a universal intra-model collaborative learning framework to enable the effective and simultaneous detection of different quality of deepfakes.
  arXiv  Detail & Related papers  (2023-09-12T02:01:31Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes [7.553507857251396]
  We propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task. SeeABLE pushes perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss. We show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities.
  arXiv  Detail & Related papers  (2022-11-21T09:38:30Z)
• Incremental-DETR: Incremental Few-Shot Object Detection via Self-Supervised Learning [60.64535309016623]
  We propose the Incremental-DETR that does incremental few-shot object detection via fine-tuning and self-supervised learning on the DETR object detector. To alleviate severe over-fitting with few novel class data, we first fine-tune the class-specific components of DETR with self-supervision. We further introduce a incremental few-shot fine-tuning strategy with knowledge distillation on the class-specific components of DETR to encourage the network in detecting novel classes without catastrophic forgetting.
  arXiv  Detail & Related papers  (2022-05-09T05:08:08Z)
• A Novel Framework for Assessment of Learning-based Detectors in Realistic Conditions with Application to Deepfake Detection [11.287342793740876]
  This paper proposes a rigorous framework to assess performance of learning-based detectors in more realistic situations. Inspired by the assessment results, a data augmentation strategy based on natural image degradation process is designed.
  arXiv  Detail & Related papers  (2022-03-22T15:03:56Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
  We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
  arXiv  Detail & Related papers  (2021-03-24T07:26:07Z)
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances [77.28192419848901]
  We propose a simple, yet effective method named contrasting shifted instances (CSI) In addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Our experiments demonstrate the superiority of our method under various novelty detection scenarios.
  arXiv  Detail & Related papers  (2020-07-16T08:32:56Z)
• Minority Class Oversampling for Tabular Data with Deep Generative Models [4.976007156860967]
  We study the ability of deep generative models to provide realistic samples that improve performance on imbalanced classification tasks via oversampling. Our experiments show that the way the method of sampling does not affect quality, but runtime varies widely. We also observe that the improvements in terms of performance metric, while shown to be significant, often are minor in absolute terms.
  arXiv  Detail & Related papers  (2020-05-07T21:35:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.