UCF: Uncovering Common Features for Generalizable Deepfake Detection

• URL: http://arxiv.org/abs/2304.13949v2
• Date: Sat, 28 Oct 2023 10:17:19 GMT
• Title: UCF: Uncovering Common Features for Generalizable Deepfake Detection
• Authors: Zhiyuan Yan, Yong Zhang, Yanbo Fan, Baoyuan Wu
• Abstract summary: Deepfake detection remains a challenging task due to the difficulty of generalizing to new types of forgeries. This paper presents a novel approach to address the two types of overfitting issues by uncovering common forgery features. Our framework can perform superior generalization than current state-of-the-art methods.
• Score: 44.12640679000489
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deepfake detection remains a challenging task due to the difficulty of generalizing to new types of forgeries. This problem primarily stems from the overfitting of existing detection methods to forgery-irrelevant features and method-specific patterns. The latter has been rarely studied and not well addressed by previous works. This paper presents a novel approach to address the two types of overfitting issues by uncovering common forgery features. Specifically, we first propose a disentanglement framework that decomposes image information into three distinct components: forgery-irrelevant, method-specific forgery, and common forgery features. To ensure the decoupling of method-specific and common forgery features, a multi-task learning strategy is employed, including a multi-class classification that predicts the category of the forgery method and a binary classification that distinguishes the real from the fake. Additionally, a conditional decoder is designed to utilize forgery features as a condition along with forgery-irrelevant features to generate reconstructed images. Furthermore, a contrastive regularization technique is proposed to encourage the disentanglement of the common and specific forgery features. Ultimately, we only utilize the common forgery features for the purpose of generalizable deepfake detection. Extensive evaluations demonstrate that our framework can perform superior generalization than current state-of-the-art methods.

Related papers

• Learning Expressive And Generalizable Motion Features For Face Forgery Detection [52.54404879581527]
  We propose an effective sequence-based forgery detection framework based on an existing video classification method. To make the motion features more expressive for manipulation detection, we propose an alternative motion consistency block. We make a general video classification network achieve promising results on three popular face forgery datasets.
  arXiv  Detail & Related papers  (2024-03-08T09:25:48Z)
• Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection [57.646582245834324]
  We propose a simple yet effective deepfake detector called LSDA. It is based on a idea: representations with a wider variety of forgeries should be able to learn a more generalizable decision boundary. We show that our proposed method is surprisingly effective and transcends state-of-the-art detectors across several widely used benchmarks.
  arXiv  Detail & Related papers  (2023-11-19T09:41:10Z)
• COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection [56.7599217711363]
  Face forgery recognition methods can only process one face at a time. Most face forgery recognition methods can only process one face at a time. We propose COMICS, an end-to-end framework for multi-face forgery detection.
  arXiv  Detail & Related papers  (2023-08-03T03:37:13Z)
• Discrepancy-Guided Reconstruction Learning for Image Forgery Detection [10.221066530624373]
  We first propose a Discrepancy-Guided (DisGE) to extract forgery-sensitive visual patterns. We then introduce a Double-Head Reconstruction (DouHR) module to enhance genuine compact visual patterns in different granular spaces. Under DouHR, we further introduce a Discrepancy-Aggregation Detector (DisAD) to aggregate these genuine compact visual patterns.
  arXiv  Detail & Related papers  (2023-04-26T07:40:43Z)
• Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection [61.74632676703288]
  We propose a novel Critical Forgery Mining framework, which can be flexibly assembled with various backbones to boost generalization and performance. Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation. We then propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses.
  arXiv  Detail & Related papers  (2023-04-24T23:02:27Z)
• SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes [7.553507857251396]
  We propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task. SeeABLE pushes perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss. We show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities.
  arXiv  Detail & Related papers  (2022-11-21T09:38:30Z)
• Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection [97.93062818228015]
  We propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block. Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field. We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video.
  arXiv  Detail & Related papers  (2021-11-17T13:30:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.