Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning

• URL: http://arxiv.org/abs/2403.07240v1
• Date: Tue, 12 Mar 2024 01:28:00 GMT
• Title: Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning
• Authors: Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, Yunchao Wei
• Abstract summary: This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
• Score: 81.98675881423131
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images despite limited training data. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. However, the rapid advancements in synthesis technology have led to specific artifacts for each generation model. Consequently, these detectors have exhibited a lack of proficiency in learning the frequency domain and tend to overfit to the artifacts present in the training data, leading to suboptimal performance on unseen sources. To address this issue, we introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors. Our method forces the detector to continuously focus on high-frequency information, exploiting high-frequency representation of features across spatial and channel dimensions. Additionally, we incorporate a straightforward frequency domain learning module to learn source-agnostic features. It involves convolutional layers applied to both the phase spectrum and amplitude spectrum between the Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (iFFT). Extensive experimentation involving 17 GANs demonstrates the effectiveness of our proposed method, showcasing state-of-the-art performance (+9.8\%) while requiring fewer parameters. The code is available at {\cred \url{https://github.com/chuangchuangtan/FreqNet-DeepfakeDetection}}.

Related papers

• Multiple Contexts and Frequencies Aggregation Network forDeepfake Detection [5.65128683992597]
  Deepfake detection faces increasing challenges since the fast growth of generative models in developing massive and diverse Deepfake technologies. Recent advances rely on introducing features from spatial or frequency domains rather than modeling general forgery features within backbones. We propose an efficient network for face forgery detection named MkfaNet, which consists of two core modules.
  arXiv  Detail & Related papers  (2024-08-03T05:34:53Z)
• Transform Once: Efficient Operator Learning in Frequency Domain [69.74509540521397]
  We study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time. This work introduces a blueprint for frequency domain learning through a single transform: transform once (T1)
  arXiv  Detail & Related papers  (2022-11-26T01:56:05Z)
• Multi-Scale Wavelet Transformer for Face Forgery Detection [43.33712402517951]
  We propose a multi-scale wavelet transformer framework for face forgery detection. Frequency-based spatial attention is designed to guide the spatial feature extractor to concentrate more on forgery traces. Cross-modality attention is proposed to fuse the frequency features with the spatial features.
  arXiv  Detail & Related papers  (2022-10-08T03:39:36Z)
• Adaptive Frequency Learning in Two-branch Face Forgery Detection [66.91715092251258]
  We propose Adaptively learn Frequency information in the two-branch Detection framework, dubbed AFD. We liberate our network from the fixed frequency transforms, and achieve better performance with our data- and task-dependent transform layers.
  arXiv  Detail & Related papers  (2022-03-27T14:25:52Z)
• Deep Frequency Filtering for Domain Generalization [55.66498461438285]
  Deep Neural Networks (DNNs) have preferences for some frequency components in the learning process. We propose Deep Frequency Filtering (DFF) for learning domain-generalizable features. We show that applying our proposed DFF on a plain baseline outperforms the state-of-the-art methods on different domain generalization tasks.
  arXiv  Detail & Related papers  (2022-03-23T05:19:06Z)
• Fourier Disentangled Space-Time Attention for Aerial Video Recognition [54.80846279175762]
  We present an algorithm, Fourier Activity Recognition (FAR), for UAV video activity recognition. Our formulation uses a novel Fourier object disentanglement method to innately separate out the human agent from the background. We have evaluated our approach on multiple UAV datasets including UAV Human RGB, UAV Human Night, Drone Action, and NEC Drone.
  arXiv  Detail & Related papers  (2022-03-21T01:24:53Z)
• FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbations [12.027711542565315]
  We design a framework to generalize the deepfake detector for both the known and unseen GAN models. Our framework generates the frequency-level perturbation maps to make the generated images indistinguishable from the real images. For experiments, we design new test scenarios varying from the training settings in GAN models, color manipulations, and object categories.
  arXiv  Detail & Related papers  (2022-02-07T16:45:11Z)
• Generalizing Face Forgery Detection with High-frequency Features [63.33397573649408]
  Current CNN-based detectors tend to overfit to method-specific color textures and thus fail to generalize. We propose to utilize the high-frequency noises for face forgery detection. The first is the multi-scale high-frequency feature extraction module that extracts high-frequency noises at multiple scales. The second is the residual-guided spatial attention module that guides the low-level RGB feature extractor to concentrate more on forgery traces from a new perspective.
  arXiv  Detail & Related papers  (2021-03-23T08:19:21Z)
• Fake Visual Content Detection Using Two-Stream Convolutional Neural Networks [14.781702606707642]
  We propose a two-stream convolutional neural network architecture called TwoStreamNet to complement frequency and spatial domain features. The proposed detector has demonstrated significant performance improvement compared to the current state-of-the-art fake content detectors.
  arXiv  Detail & Related papers  (2021-01-03T18:05:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.