Balancing Frequencies and Pixels in Flow Matching

• URL: http://arxiv.org/abs/2609.02748v1
• Date: Wed, 02 Sep 2026 15:49:24 GMT
• Title: Balancing Frequencies and Pixels in Flow Matching
• Abstract summary: Pixel-space reconstruction objectives treat all spatial errors uniformly, causing low frequencies to dominate the optimization signal.<n>We propose a Focal Log-Frequency Loss objective that equalizes the learning signal across frequencies.<n>Our approach is conceptually simple, requires no architectural changes, and acts as a drop-in replacement for flow matching losses.
• Score: 43.75072807799361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural images follow a $1/f^2$ spectral distribution: most signal energy lies in the low spatial frequencies, while the perceptually important structures such as textures and edges occupy sparse high-frequency bands. Pixel-space reconstruction objectives, however, treat all spatial errors uniformly, causing low frequencies to dominate the optimization signal and delaying the learning of fine-scale details. In this work, we identify this objective-level spectral imbalance as a key inefficiency in training pixel-space flow models. To address it, we propose a Focal Log-Frequency Loss (f-loss), a spectrally balanced objective that equalizes the learning signal across frequencies, emphasizing high-frequency components that are otherwise underrepresented in pixel-space objectives. Building on this, we introduce a simple training strategy that combines frequency and pixel supervision: we first emphasize frequency-domain learning early to capture all frequencies, and then transition to standard pixel-space v-loss for spatial refinement. This balancing mitigates the low-frequency bias of pixel losses and aligns the training signal with the evolving needs of the model. Our approach is conceptually simple, requires no architectural changes, and acts as a drop-in replacement for flow matching losses. Across multiple model scales, it accelerates convergence by up to 40% while consistently improving FID and perceptual fidelity. We will release code and models.

Related papers

• WaiT for the Signal: Simple Frequency-Aware Flow-Matching [60.56691023057048]
  We introduce WaiT, a Wavelet-aware image Transformer that decomposes generation into coarse and fine bands.<n>On ImageNet 512x512, WaiT achieves a pixel-space FID of 1.43, reducing sampling compute by up to 50%.<n>Our formulation outperforms even the strongest latent-space models on texture fidelity, and scales seamlessly to high-resolution OpenImages and to video generation.
  arXiv  Detail & Related papers  (2026-07-30T18:26:19Z)
• Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion [46.51681899437508]
  We introduce Spectral Forcing, a parameter-free, time-conditional 2D-DCT low-pass operator applied to the noisy input before the patch embedder.<n>Its cutoff expands monotonically with the diffusion time and becomes the identity at the data endpoint.<n>On ImageNet-256 with JiT-700M/32, Spectral Forcing consistently improves both FID and Inception Score across different training epochs.
  arXiv  Detail & Related papers  (2026-06-13T10:29:11Z)
• Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations [7.7579389797716365]
  We propose an adaptive local frequency filtering method for Fourier-encoded INRs.<n>The proposed method introduces a spatially varying parameter $(mathbfx)$ to modulate encoded Fourier components.<n>Experiments on 2D image fitting, 3D shape representation, and sparse data reconstruction demonstrate that the proposed method consistently improves reconstruction quality.
  arXiv  Detail & Related papers  (2026-04-03T08:04:30Z)
• Spectral Bottleneck in Deep Neural Networks: Noise is All You Need [0.0]
  We study the challenge of fitting high-frequency-dominant signals susceptible to spectral bottleneck.<n>To effectively fit any target signal irrespective of it's frequency content, we propose a generalized target perturbation scheme.<n>We show that the noise scales can provide control over the spectra of network activations and the eigenbasis of the empirical neural tangent kernel.
  arXiv  Detail & Related papers  (2025-09-09T22:16:24Z)
• Freqformer: Image-Demoiréing Transformer via Efficient Frequency Decomposition [83.40450475728792]
  We present Freqformer, a Transformer-based framework specifically designed for image demoir'eing through targeted frequency separation.<n>Our method performs an effective frequency decomposition that explicitly splits moir'e patterns into high-frequency spatially-localized textures and low-frequency scale-robust color distortions.<n>Experiments on various demoir'eing benchmarks demonstrate that Freqformer achieves state-of-the-art performance with a compact model size.
  arXiv  Detail & Related papers  (2025-05-25T12:23:10Z)
• FreqU-FNet: Frequency-Aware U-Net for Imbalanced Medical Image Segmentation [0.0]
  FreqU-FNet is a novel U-shaped segmentation architecture operating in the frequency domain.<n>Our framework incorporates a Frequency that leverages Low-Pass Convolution and Daubechies wavelet-based downsampling.<n>Experiments on multiple medical segmentation benchmarks demonstrate that FreqU-FNet consistently outperforms both CNN and Transformer baselines.
  arXiv  Detail & Related papers  (2025-05-23T06:51:24Z)
• Misalignment-Robust Frequency Distribution Loss for Image Transformation [51.0462138717502]
  This paper aims to address a common challenge in deep learning-based image transformation methods, such as image enhancement and super-resolution. We introduce a novel and simple Frequency Distribution Loss (FDL) for computing distribution distance within the frequency domain. Our method is empirically proven effective as a training constraint due to the thoughtful utilization of global information in the frequency domain.
  arXiv  Detail & Related papers  (2024-02-28T09:27:41Z)
• Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Models [89.76587063609806]
  We study the denoising diffusion probabilistic model (DDPM) in wavelet space, instead of pixel space, for visual synthesis. By explicitly modeling the wavelet signals, we find our model is able to generate images with higher quality on several datasets.
  arXiv  Detail & Related papers  (2023-07-27T06:53:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.