Beyond Spatial Frequency: Pixel-wise Temporal Frequency-based Deepfake Video Detection

• URL: http://arxiv.org/abs/2507.02398v2
• Date: Thu, 10 Jul 2025 06:52:35 GMT
• Title: Beyond Spatial Frequency: Pixel-wise Temporal Frequency-based Deepfake Video Detection
• Authors: Taehoon Kim, Jongwook Choi, Yonghyun Jeong, Haeun Noh, Jaejun Yoo, Seungryul Baek, Jongwon Choi,
• Abstract summary: We introduce a deepfake video detection approach that exploits pixel-wise temporal inconsistencies.<n>Our approach performs a 1D spectra transform on the time for each pixel, extracting features highly sensitive to temporal inconsistencies.<n>Our framework represents a significant advancement in deepfake video detection providing robust performance across diverse and challenging detection scenarios.
• Score: 20.96211739806439
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a deepfake video detection approach that exploits pixel-wise temporal inconsistencies, which traditional spatial frequency-based detectors often overlook. Traditional detectors represent temporal information merely by stacking spatial frequency spectra across frames, resulting in the failure to detect temporal artifacts in the pixel plane. Our approach performs a 1D Fourier transform on the time axis for each pixel, extracting features highly sensitive to temporal inconsistencies, especially in areas prone to unnatural movements. To precisely locate regions containing the temporal artifacts, we introduce an attention proposal module trained in an end-to-end manner. Additionally, our joint transformer module effectively integrates pixel-wise temporal frequency features with spatio-temporal context features, expanding the range of detectable forgery artifacts. Our framework represents a significant advancement in deepfake video detection, providing robust performance across diverse and challenging detection scenarios.

Related papers

• Track Any Anomalous Object: A Granular Video Anomaly Detection Pipeline [63.96226274616927]
  A new framework called Track Any Anomalous Object (TAO) introduces a granular video anomaly detection pipeline.<n>Unlike methods that assign anomaly scores to every pixel, our approach transforms the problem into pixel-level tracking of anomalous objects.<n>Experiments demonstrate that TAO sets new benchmarks in accuracy and robustness.
  arXiv  Detail & Related papers  (2025-06-05T15:49:39Z)
• Detecting Inpainted Video with Frequency Domain Insights [0.0]
  We propose the Frequency Domain Insights Network (FDIN), which significantly enhances detection accuracy.<n>Previous evaluations on public datasets demonstrate that FDIN achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-09-21T01:51:07Z)
• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• Multimodal Graph Learning for Deepfake Detection [10.077496841634135]
  Existing deepfake detectors face several challenges in achieving robustness and generalization. We propose a novel framework, namely Multimodal Graph Learning (MGL), that leverages information from multiple modalities. Our proposed method aims to effectively identify and utilize distinguishing features for deepfake detection.
  arXiv  Detail & Related papers  (2022-09-12T17:17:49Z)
• Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in VIS and NIR Scenario [87.72258480670627]
  Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum compared to the real images. This paper proposes a Cosine Transform-based Forgery Clue Augmentation Network (FCAN-DCT) to achieve a more comprehensive spatial-temporal feature representation.
  arXiv  Detail & Related papers  (2022-07-05T09:27:53Z)
• Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain [88.7339322596758]
  We present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery. SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
  arXiv  Detail & Related papers  (2021-03-02T16:45:08Z)
• Deep Video Inpainting Detection [95.36819088529622]
  Video inpainting detection localizes an inpainted region in a video both spatially and temporally. VIDNet, Video Inpainting Detection Network, contains a two-stream encoder-decoder architecture with attention module.
  arXiv  Detail & Related papers  (2021-01-26T20:53:49Z)
• Spatio-temporal Features for Generalized Detection of Deepfake Videos [12.453288832098314]
  We propose-temporal features, modeled by 3D CNNs, to extend the capabilities to detect new sorts of deep videos. We show that our approach outperforms existing methods in terms of generalization capabilities.
  arXiv  Detail & Related papers  (2020-10-22T16:28:50Z)
• Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Prediction [12.84409065286371]
  We propose a video prediction network based on multi-level wavelet analysis to deal with spatial and temporal information in a unified manner. Our model shows significant improvements on fidelity and temporal consistency over state-of-the-art works.
  arXiv  Detail & Related papers  (2020-02-23T13:46:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.