Understanding Why Foundation Models Work for Diffusion-Generated Image Detection

• URL: http://arxiv.org/abs/2608.12155v1
• Date: Wed, 12 Aug 2026 15:18:10 GMT
• Title: Understanding Why Foundation Models Work for Diffusion-Generated Image Detection
• Abstract summary: We investigate what cues are exploited by foundation-model-based detectors to distinguish real images from diffusion-generated ones.<n>Our results suggest that foundation-model-based detectors succeed by capturing non-semantic low-to-mid frequency distributional discrepancies between real and diffusion-generated images.
• Score: 17.35112221493337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision foundation models have recently emerged as powerful feature extractors for detecting AI-generated images, achieving strong generalization across generators and robustness to common image degradations. However, the reason behind their effectiveness is poorly understood. In this work, we investigate what cues are exploited by foundation-model-based detectors to distinguish real images from diffusion-generated ones. To this end, we design an ad hoc analysis protocol based on DDIM inversion. Given a real image we generate a sequence of synthetic copies by changing the depth of DDIM inversion. Even though most copies are semantically identical to the real reference, the detector score varies significantly across them due to subtle traces introduced by the diffusion synthesis, showing that its decision is not primarily driven by semantic failures. Through a frequency-swapping analysis, we further reveal that the discriminative cues exploited by the detectors are mainly localized in the low-to-mid frequency range, rather than only in the high-frequency range, as is the case for artifacts commonly associated with generative models. Finally, a latent-space analysis shows that regenerated images exhibit reduced variance and effective dimensionality, indicating that diffusion models do not fully reproduce the variability of real data. Overall, our results suggest that foundation-model-based detectors succeed by capturing non-semantic low-to-mid frequency distributional discrepancies between real and diffusion-generated images. These findings provide new insight into the robustness and generalization of such detectors and suggest directions for more interpretable forensic methods.

Related papers

• FIND: A Simple yet Effective Baseline for Diffusion-Generated Image Detection [32.11320589253777]
  Real images are more difficult to fit with Gaussian distributions compared to synthetic ones.<n>We propose Forgery Identification via Noise Disturbance (FIND), a novel method that requires only a simple binary classifier.<n>FIND improves performance by 11.7% on the GenImage benchmark while running 126x faster than existing methods.
  arXiv  Detail & Related papers  (2026-03-15T04:49:30Z)
• Detecting AI-Generated Images via Diffusion Snap-Back Reconstruction: A Forensic Approach [0.0]
  Traditional deepfake detection methods fail against text-to-image systems such as Stable Diffusion and DALL-E.<n>This paper introduces a diffusion-based forensic framework that leverages multi-strength image reconstruction dynamics.
  arXiv  Detail & Related papers  (2025-11-01T01:35:54Z)
• Who Made This? Fake Detection and Source Attribution with Diffusion Features [0.15293427903448018]
  We introduce FRIDA, a framework for deepfake detection and source attribution.<n>A compact neural model enables accurate source attribution.<n>Results show that diffusion representations inherently encode generator-specific patterns.
  arXiv  Detail & Related papers  (2025-10-31T16:27:34Z)
• Explainable Synthetic Image Detection through Diffusion Timestep Ensembling [30.298198387824275]
  We propose a novel synthetic image detection method that directly utilizes features of intermediately noised images by training an ensemble on multiple noised timesteps.<n>To enhance human comprehension, we introduce a metric-grounded explanation generation and refinement module.<n>Our method achieves state-of-the-art performance with 98.91% and 95.89% detection accuracy on regular and challenging samples respectively.
  arXiv  Detail & Related papers  (2025-03-08T13:04:20Z)
• DiffDoctor: Diagnosing Image Diffusion Models Before Treating [57.82359018425674]
  We propose DiffDoctor, a two-stage pipeline to assist image diffusion models in generating fewer artifacts.<n>We collect a dataset of over 1M flawed synthesized images and set up an efficient human-in-the-loop annotation process.<n>The learned artifact detector is then involved in the second stage to optimize the diffusion model by providing pixel-level feedback.
  arXiv  Detail & Related papers  (2025-01-21T18:56:41Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection [86.97062579515833]
  We introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by tft28 distinct generative models. This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable tft11.6% improvement over existing methods.
  arXiv  Detail & Related papers  (2023-12-16T14:27:06Z)
• Projection Regret: Reducing Background Bias for Novelty Detection via Diffusion Models [72.07462371883501]
  We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information. PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality. Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
  arXiv  Detail & Related papers  (2023-12-05T09:44:47Z)
• Detecting Images Generated by Deep Diffusion Models using their Local Intrinsic Dimensionality [8.968599131722023]
  Diffusion models have been successfully applied for the visual synthesis of strikingly realistic appearing images. This raises strong concerns about their potential for malicious purposes. We propose using the lightweight multi Local Intrinsic Dimensionality (multiLID) for the automatic detection of synthetic images.
  arXiv  Detail & Related papers  (2023-07-05T15:03:10Z)
• DIRE for Diffusion-Generated Image Detection [128.95822613047298]
  We propose a novel representation called DIffusion Reconstruction Error (DIRE) DIRE measures the error between an input image and its reconstruction counterpart by a pre-trained diffusion model. It provides a hint that DIRE can serve as a bridge to distinguish generated and real images.
  arXiv  Detail & Related papers  (2023-03-16T13:15:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.