A Fourier Space Perspective on Diffusion Models

• URL: http://arxiv.org/abs/2505.11278v1
• Date: Fri, 16 May 2025 14:13:02 GMT
• Title: A Fourier Space Perspective on Diffusion Models
• Authors: Fabian Falck, Teodora Pandeva, Kiarash Zahirnia, Rachel Lawrence, Richard Turner, Edward Meeds, Javier Zazo, Sushrut Karmalkar,
• Abstract summary: Diffusion models are state-of-the-art generative models on data modalities such as images, audio, proteins and materials.<n>We study the inductive bias of the forward process of diffusion models in Fourier space.
• Score: 6.834230686279937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models are state-of-the-art generative models on data modalities such as images, audio, proteins and materials. These modalities share the property of exponentially decaying variance and magnitude in the Fourier domain. Under the standard Denoising Diffusion Probabilistic Models (DDPM) forward process of additive white noise, this property results in high-frequency components being corrupted faster and earlier in terms of their Signal-to-Noise Ratio (SNR) than low-frequency ones. The reverse process then generates low-frequency information before high-frequency details. In this work, we study the inductive bias of the forward process of diffusion models in Fourier space. We theoretically analyse and empirically demonstrate that the faster noising of high-frequency components in DDPM results in violations of the normality assumption in the reverse process. Our experiments show that this leads to degraded generation quality of high-frequency components. We then study an alternate forward process in Fourier space which corrupts all frequencies at the same rate, removing the typical frequency hierarchy during generation, and demonstrate marked performance improvements on datasets where high frequencies are primary, while performing on par with DDPM on standard imaging benchmarks.

Related papers

• NFCDS: A Plug-and-Play Noise Frequency-Controlled Diffusion Sampling Strategy for Image Restoration [20.351955950047348]
  Diffusion-based Plug-and-Play (NFC) methods produce images with high quality but often suffer from reduced fidelity data.<n>We propose Frequency Diffusion-led Sampling (NFCDS), a modulation mechanism for reverse diffusion noise.
  arXiv  Detail & Related papers  (2026-01-29T04:10:45Z)
• Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learning [10.270002679268485]
  We propose a novel diffusion-based RL framework that integrates Discrete Wavelet Transform to decompose trajectories into low- and high-frequency components.<n>WFDiffuser effectively mitigates frequency shift, leading to smoother, more stable trajectories and improved decision-making performance.
  arXiv  Detail & Related papers  (2025-09-04T08:50:31Z)
• Frequency Regulation for Exposure Bias Mitigation in Diffusion Models [13.095683155232281]
  We make a key observation: the energy of predicted noisy samples in the reverse process continuously declines compared to perturbed samples in the forward process.<n>We introduce a dynamic frequency regulation mechanism utilizing wavelet transforms, which separately adjusts the low- and high-frequency subbands.<n>We derive the rigorous mathematical form of exposure bias.
  arXiv  Detail & Related papers  (2025-07-14T08:58:38Z)
• Noise Conditional Variational Score Distillation [60.38982038894823]
  Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
  arXiv  Detail & Related papers  (2025-06-11T06:01:39Z)
• Fredformer: Frequency Debiased Transformer for Time Series Forecasting [8.356290446630373]
  The Transformer model has shown leading performance in time series forecasting. It tends to learn low-frequency features in the data and overlook high-frequency features, showing a frequency bias. We propose Fredformer, a framework designed to mitigate frequency bias by learning features equally across different frequency bands.
  arXiv  Detail & Related papers  (2024-06-13T11:29:21Z)
• Boosting Diffusion Models with Moving Average Sampling in Frequency Domain [101.43824674873508]
  Diffusion models rely on the current sample to denoise the next one, possibly resulting in denoising instability. In this paper, we reinterpret the iterative denoising process as model optimization and leverage a moving average mechanism to ensemble all the prior samples. We name the complete approach "Moving Average Sampling in Frequency domain (MASF)"
  arXiv  Detail & Related papers  (2024-03-26T16:57:55Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• PartDiff: Image Super-resolution with Partial Diffusion Models [3.8435187580887717]
  Denoising diffusion probabilistic models (DDPMs) have achieved impressive performance on various image generation tasks. DDPMs generate new data by iteratively denoising from random noise. But diffusion-based generative models suffer from high computational costs due to the large number of denoising steps. This paper proposes the Partial Diffusion Model (PartDiff), which diffuses the image to an intermediate latent state instead of pure random noise.
  arXiv  Detail & Related papers  (2023-07-21T22:11:23Z)
• Modeling low- and high-frequency noise in transmon qubits with resource-efficient measurement [0.0]
  Transmon qubits experience open system effects that manifest as noise at a broad range of frequencies. We present a model of these effects using the Redfield master equation with a hybrid bath consisting of low and high-frequency components. We use two-level fluctuators to simulate 1/f-like noise behavior, which is a dominant source of decoherence for superconducting qubits.
  arXiv  Detail & Related papers  (2023-02-28T21:46:03Z)
• Diffusion Probabilistic Model Made Slim [128.2227518929644]
  We introduce a customized design for slim diffusion probabilistic models (DPM) for light-weight image synthesis. We achieve 8-18x computational complexity reduction as compared to the latent diffusion models on a series of conditional and unconditional image generation tasks.
  arXiv  Detail & Related papers  (2022-11-27T16:27:28Z)
• Transform Once: Efficient Operator Learning in Frequency Domain [69.74509540521397]
  We study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time. This work introduces a blueprint for frequency domain learning through a single transform: transform once (T1)
  arXiv  Detail & Related papers  (2022-11-26T01:56:05Z)
• Accelerating Diffusion Models via Early Stop of the Diffusion Process [114.48426684994179]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved impressive performance on various generation tasks. In practice DDPMs often need hundreds even thousands of denoising steps to obtain a high-quality sample. We propose a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), for DDPMs.
  arXiv  Detail & Related papers  (2022-05-25T06:40:09Z)
• SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping [51.698273019061645]
  SpecGrad adapts the diffusion noise so that its time-varying spectral envelope becomes close to the conditioning log-mel spectrogram. It is processed in the time-frequency domain to keep the computational cost almost the same as the conventional DDPM-based neural vocoders.
  arXiv  Detail & Related papers  (2022-03-31T02:08:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.