DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion

• URL: http://arxiv.org/abs/2410.04372v1
• Date: Sun, 6 Oct 2024 06:22:43 GMT
• Title: DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion
• Authors: Ke Sun, Shen Chen, Taiping Yao, Hong Liu, Xiaoshuai Sun, Shouhong Ding, Rongrong Ji,
• Abstract summary: Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
• Score: 94.46904504076124
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The rapid progress of Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. In this paper, we revisit the generation process and identify a universal principle: Deepfake images inherently contain information from both source and target identities, while genuine faces maintain a consistent identity. Building upon this insight, we introduce DiffusionFake, a novel plug-and-play framework that reverses the generative process of face forgeries to enhance the generalization of detection models. DiffusionFake achieves this by injecting the features extracted by the detection model into a frozen pre-trained Stable Diffusion model, compelling it to reconstruct the corresponding target and source images. This guided reconstruction process constrains the detection network to capture the source and target related features to facilitate the reconstruction, thereby learning rich and disentangled representations that are more resilient to unseen forgeries. Extensive experiments demonstrate that DiffusionFake significantly improves cross-domain generalization of various detector architectures without introducing additional parameters during inference. Our Codes are available in https://github.com/skJack/DiffusionFake.git.

Related papers

• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis [71.40724659748787]
  DiffusionFace is the first diffusion-based face forgery dataset. It covers various forgery categories, including unconditional and Text Guide facial image generation, Img2Img, Inpaint, and Diffusion-based facial exchange algorithms. It provides essential metadata and a real-world internet-sourced forgery facial image dataset for evaluation.
  arXiv  Detail & Related papers  (2024-03-27T11:32:44Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Recap: Detecting Deepfake Video with Unpredictable Tampered Traces via Recovering Faces and Mapping Recovered Faces [35.04806736119123]
  We propose Recap, a novel Deepfake detection model that exposes unspecific facial part inconsistencies by recovering faces. In the recovering stage, the model focuses on randomly masking regions of interest and reconstructing real faces without unpredictable tampered traces. In the mapping stage, the output of the recovery phase serves as supervision to guide the facial mapping process.
  arXiv  Detail & Related papers  (2023-08-19T06:18:11Z)
• On the Vulnerability of DeepFake Detectors to Attacks Generated by Denoising Diffusion Models [0.5827521884806072]
  We investigate the vulnerability of single-image deepfake detectors to black-box attacks created by the newest generation of generative methods. Our experiments are run on FaceForensics++, a widely used deepfake benchmark consisting of manipulated images. Our findings indicate that employing just a single denoising diffusion step in the reconstruction process of a deepfake can significantly reduce the likelihood of detection.
  arXiv  Detail & Related papers  (2023-07-11T15:57:51Z)
• Real Face Foundation Representation Learning for Generalized Deepfake Detection [74.4691295738097]
  The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is almost impossible to collect sufficient representative fake faces, and it is hard for existing detectors to generalize to all types of manipulation. We propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets.
  arXiv  Detail & Related papers  (2023-03-15T08:27:56Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.