Deepfake Detection without Deepfakes: Generalization via Synthetic Frequency Patterns Injection

• URL: http://arxiv.org/abs/2403.13479v1
• Date: Wed, 20 Mar 2024 10:33:10 GMT
• Title: Deepfake Detection without Deepfakes: Generalization via Synthetic Frequency Patterns Injection
• Authors: Davide Alessandro Coccomini, Roberto Caldelli, Claudio Gennaro, Giuseppe Fiameni, Giuseppe Amato, Fabrizio Falchi,
• Abstract summary: Deepfake detectors are typically trained on large sets of pristine and generated images. Deepfake detectors excel at identifying deepfakes created through methods encountered during training but struggle with those generated by unknown techniques. This paper introduces a learning approach aimed at significantly enhancing the generalization capabilities of deepfake detectors.
• Score: 12.33030785907372
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deepfake detectors are typically trained on large sets of pristine and generated images, resulting in limited generalization capacity; they excel at identifying deepfakes created through methods encountered during training but struggle with those generated by unknown techniques. This paper introduces a learning approach aimed at significantly enhancing the generalization capabilities of deepfake detectors. Our method takes inspiration from the unique "fingerprints" that image generation processes consistently introduce into the frequency domain. These fingerprints manifest as structured and distinctly recognizable frequency patterns. We propose to train detectors using only pristine images injecting in part of them crafted frequency patterns, simulating the effects of various deepfake generation techniques without being specific to any. These synthetic patterns are based on generic shapes, grids, or auras. We evaluated our approach using diverse architectures across 25 different generation methods. The models trained with our approach were able to perform state-of-the-art deepfake detection, demonstrating also superior generalization capabilities in comparison with previous methods. Indeed, they are untied to any specific generation technique and can effectively identify deepfakes regardless of how they were made.

Related papers

• Frequency Masking for Universal Deepfake Detection [25.844830329275613]
  We study universal deepfake detection. Our goal is to detect synthetic images from a range of generative AI approaches. Motivated by recently proposed masked image modeling, we make the first attempt to explore masked image modeling for universal deepfake detection.
  arXiv  Detail & Related papers  (2024-01-12T11:02:12Z)
• Detecting Generated Images by Real Images Only [64.12501227493765]
  Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training. This paper approaches the generated image detection problem from a new perspective: Start from real images. By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
  arXiv  Detail & Related papers  (2023-11-02T03:09:37Z)
• How Generalizable are Deepfake Image Detectors? An Empirical Study [4.42204674141385]
  We present the first empirical study on the generalizability of deepfake detectors. Our study utilizes six deepfake datasets, five deepfake image detection methods, and two model augmentation approaches. We find that detectors are learning unwanted properties specific to synthesis methods and struggling to extract discriminative features.
  arXiv  Detail & Related papers  (2023-08-08T10:30:34Z)
• Misleading Deep-Fake Detection with GAN Fingerprints [14.459389888856412]
  We show that an adversary can remove indicative artifacts, the GAN fingerprint, directly from the frequency spectrum of a generated image. Our results show that an adversary can often remove GAN fingerprints and thus evade the detection of generated images.
  arXiv  Detail & Related papers  (2022-05-25T07:32:12Z)
• Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
  The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
  arXiv  Detail & Related papers  (2022-04-06T20:51:40Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)
• TAR: Generalized Forensic Framework to Detect Deepfakes using Weakly Supervised Learning [17.40885531847159]
  Deepfakes have become a critical social problem, and detecting them is of utmost importance. In this work, we introduce a practical digital forensic tool to detect different types of deepfakes simultaneously. We develop an autoencoder-based detection model with Residual blocks and sequentially perform transfer learning to detect different types of deepfakes simultaneously.
  arXiv  Detail & Related papers  (2021-05-13T07:31:08Z)
• Deepfake Forensics via An Adversarial Game [99.84099103679816]
  We advocate adversarial training for improving the generalization ability to both unseen facial forgeries and unseen image/video qualities. Considering that AI-based face manipulation often leads to high-frequency artifacts that can be easily spotted by models yet difficult to generalize, we propose a new adversarial training method that attempts to blur out these specific artifacts.
  arXiv  Detail & Related papers  (2021-03-25T02:20:08Z)
• A Convolutional LSTM based Residual Network for Deepfake Video Detection [23.275080108063406]
  We develop a Convolutional LSTM based Residual Network (CLRNet) to detect deepfake videos. We also propose a transfer learning-based approach to generalize different deepfake methods.
  arXiv  Detail & Related papers  (2020-09-16T05:57:06Z)
• Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data [64.65952078807086]
  Photorealistic image generation has reached a new level of quality due to the breakthroughs of generative adversarial networks (GANs) Yet, the dark side of such deepfakes, the malicious use of generated media, raises concerns about visual misinformation. We seek a proactive and sustainable solution on deepfake detection by introducing artificial fingerprints into the models.
  arXiv  Detail & Related papers  (2020-07-16T16:49:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.