A Quality-Centric Framework for Generic Deepfake Detection

• URL: http://arxiv.org/abs/2411.05335v2
• Date: Tue, 26 Nov 2024 06:37:21 GMT
• Title: A Quality-Centric Framework for Generic Deepfake Detection
• Authors: Wentang Song, Zhiyuan Yan, Yuzhen Lin, Taiping Yao, Changsheng Chen, Shen Chen, Yandan Zhao, Shouhong Ding, Bin Li,
• Abstract summary: forgery quality of different deepfakes varies: some have easily recognizable forgery clues, while others are highly realistic. Existing works often train detectors on a mix of deepfakes with varying forgery qualities, leading detectors to short-cut the easy-to-spot artifacts from low-quality forgery samples. We propose a novel quality-centric framework for generic deepfake detection, which is composed of a Quality Evaluator, a low-quality data enhancement module, and a learning pacing strategy.
• Score: 41.1730636042585
• License:
• Abstract: This paper addresses the generalization issue in deepfake detection by harnessing forgery quality in training data. Generally, the forgery quality of different deepfakes varies: some have easily recognizable forgery clues, while others are highly realistic. Existing works often train detectors on a mix of deepfakes with varying forgery qualities, potentially leading detectors to short-cut the easy-to-spot artifacts from low-quality forgery samples, thereby hurting generalization performance. To tackle this issue, we propose a novel quality-centric framework for generic deepfake detection, which is composed of a Quality Evaluator, a low-quality data enhancement module, and a learning pacing strategy that explicitly incorporates forgery quality into the training process. The framework is inspired by curriculum learning, which is designed to gradually enable the detector to learn more challenging deepfake samples, starting with easier samples and progressing to more realistic ones. We employ both static and dynamic assessments to assess the forgery quality, combining their scores to produce a final rating for each training sample. The rating score guides the selection of deepfake samples for training, with higher-rated samples having a higher probability of being chosen. Furthermore, we propose a novel frequency data augmentation method specifically designed for low-quality forgery samples, which helps to reduce obvious forgery traces and improve their overall realism. Extensive experiments show that our method can be applied in a plug-and-play manner and significantly enhance the generalization performance.

Related papers

• Towards General Deepfake Detection with Dynamic Curriculum [4.622705420257596]
  We propose to introduce the sample hardness into the training of deepfake detectors via the curriculum learning paradigm. We present a novel simple yet effective strategy, named Dynamic Facial Forensic Curriculum (DFFC), which makes the model gradually focus on hard samples during the training. Comprehensive experiments show that DFFC can improve both within- and cross-dataset performance of various kinds of end-to-end deepfake detectors.
  arXiv  Detail & Related papers  (2024-10-15T00:58:09Z)
• Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection [15.857961926916465]
  We present a novel general deepfake detection method, called textbfCurricular textbfDynamic textbfForgery textbfAugmentation (CDFA) CDFA jointly trains a deepfake detector with a forgery augmentation policy network. We show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors.
  arXiv  Detail & Related papers  (2024-09-22T13:51:22Z)
• Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models [52.04189118767758]
  Generalization is a main issue for current audio deepfake detectors. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection.
  arXiv  Detail & Related papers  (2024-05-03T15:27:11Z)
• Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learning [26.517887637150594]
  Deepfake has recently raised a plethora of societal concerns over its possible security threats and dissemination of fake information. Most SOTA approaches are limited by using a single specific model for detecting certain deepfake video quality type. We propose a universal intra-model collaborative learning framework to enable the effective and simultaneous detection of different quality of deepfakes.
  arXiv  Detail & Related papers  (2023-09-12T02:01:31Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• A Novel Framework for Assessment of Learning-based Detectors in Realistic Conditions with Application to Deepfake Detection [11.287342793740876]
  This paper proposes a rigorous framework to assess performance of learning-based detectors in more realistic situations. Inspired by the assessment results, a data augmentation strategy based on natural image degradation process is designed.
  arXiv  Detail & Related papers  (2022-03-22T15:03:56Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
  We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
  arXiv  Detail & Related papers  (2021-03-24T07:26:07Z)
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances [77.28192419848901]
  We propose a simple, yet effective method named contrasting shifted instances (CSI) In addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Our experiments demonstrate the superiority of our method under various novelty detection scenarios.
  arXiv  Detail & Related papers  (2020-07-16T08:32:56Z)
• Minority Class Oversampling for Tabular Data with Deep Generative Models [4.976007156860967]
  We study the ability of deep generative models to provide realistic samples that improve performance on imbalanced classification tasks via oversampling. Our experiments show that the way the method of sampling does not affect quality, but runtime varies widely. We also observe that the improvements in terms of performance metric, while shown to be significant, often are minor in absolute terms.
  arXiv  Detail & Related papers  (2020-05-07T21:35:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.