Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models

• URL: http://arxiv.org/abs/2309.02218v1
• Date: Tue, 5 Sep 2023 13:22:41 GMT
• Title: Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models
• Authors: Haixu Song, Shiyu Huang, Yinpeng Dong, Wei-Wei Tu
• Abstract summary: We present an investigation into how deepfakes are produced and how they can be identified. The cornerstone of our research is a rich collection of artificial celebrity faces, titled DeepFakeFace. This data serves as a robust foundation to train and test algorithms designed to spot deepfakes.
• Score: 35.188364409869465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rise of deepfake images, especially of well-known personalities, poses a serious threat to the dissemination of authentic information. To tackle this, we present a thorough investigation into how deepfakes are produced and how they can be identified. The cornerstone of our research is a rich collection of artificial celebrity faces, titled DeepFakeFace (DFF). We crafted the DFF dataset using advanced diffusion models and have shared it with the community through online platforms. This data serves as a robust foundation to train and test algorithms designed to spot deepfakes. We carried out a thorough review of the DFF dataset and suggest two evaluation methods to gauge the strength and adaptability of deepfake recognition tools. The first method tests whether an algorithm trained on one type of fake images can recognize those produced by other methods. The second evaluates the algorithm's performance with imperfect images, like those that are blurry, of low quality, or compressed. Given varied results across deepfake methods and image changes, our findings stress the need for better deepfake detectors. Our DFF dataset and tests aim to boost the development of more effective tools against deepfakes.

Related papers

• AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors [24.78672820633581]
  Deep generative models can create remarkably fake images while raising concerns about misinformation and copyright infringement. Deepfake detection technique is developed to distinguish between real and fake images. We propose a novel approach called AntifakePrompt, using Vision-Language Models and prompt tuning techniques.
  arXiv  Detail & Related papers  (2023-10-26T14:23:45Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• Fooling State-of-the-Art Deepfake Detection with High-Quality Deepfakes [2.0883760606514934]
  We show that deepfake detectors proven to generalize well on multiple research datasets still struggle in real-world scenarios with well-crafted fakes. We propose a novel autoencoder for face swapping alongside an advanced face blending technique, which we utilize to generate 90 high-quality deepfakes.
  arXiv  Detail & Related papers  (2023-05-09T09:08:49Z)
• Deepfake Detection of Occluded Images Using a Patch-based Approach [1.6114012813668928]
  We present a deep learning approach using the entire face and face patches to distinguish real/fake images in the presence of obstruction. For producing fake images, StyleGAN and StyleGAN2 are trained by FFHQ images and also StarGAN and PGGAN are trained by CelebA images. The proposed approach reaches higher results in early epochs than other methods and increases the SoTA results by 0.4%-7.9% in the different built data-sets.
  arXiv  Detail & Related papers  (2023-04-10T12:12:14Z)
• Real Face Foundation Representation Learning for Generalized Deepfake Detection [74.4691295738097]
  The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is almost impossible to collect sufficient representative fake faces, and it is hard for existing detectors to generalize to all types of manipulation. We propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets.
  arXiv  Detail & Related papers  (2023-03-15T08:27:56Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• An Experimental Evaluation on Deepfake Detection using Deep Face Recognition [0.0]
  Deep learning has led to the generation of very realistic fake content, also known as deepfakes. Most of the current deepfake detection methods are deemed as a binary classification problem in distinguishing authentic images or videos from fake ones using two-class convolutional neural networks (CNNs) This paper thoroughly evaluate the efficacy of deep face recognition in identifying deepfakes, using different loss functions and deepfake generation techniques.
  arXiv  Detail & Related papers  (2021-10-04T18:02:56Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
  We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
  arXiv  Detail & Related papers  (2021-01-05T11:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.