FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion

• URL: http://arxiv.org/abs/2605.17759v1
• Date: Mon, 18 May 2026 02:25:07 GMT
• Title: FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion
• Abstract summary: FrequencyBooster is a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead.<n>Our model achieves a state-of-the-art FID of textbf1.60 at $256 times 256$ resolution within only 320 epochs.<n>At $512 times 512$ resolution, FrequencyBooster attains an FID of textbf1.69, significantly outperforming existing pixel-space and latent-space generative models.
• Score: 15.557373713923306
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: To circumvent the inherent fidelity bottlenecks and optimization misalignment of VAE-based latent diffusion, pixel-space diffusion models have emerged as a compelling end-to-end paradigm. However, existing pixel diffusion models often struggle to balance computational efficiency with the preservation of high-frequency details. They frequently resort to patch-based compression or restricted local decoding, leading to a "spectral compromise" where high-frequency and fine-grained pixel information are suppressed. To address these challenges, we propose \textbf{FrequencyBooster}, a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead. The core of our method is a high-capacity decoder that specializes in extracting exhaustive high-frequency details and low-frequency semantics, the latter of which is derived from a Diffusion Transformer (DiT) backbone. Unlike prior works that sacrifice global context for local refinement, FrequencyBooster leverages high-dimensional feature representations to maintain global structural integrity while achieving superior pixel-level precision. Extensive experiments on ImageNet demonstrate the effectiveness of our approach: our model achieves a state-of-the-art FID of \textbf{1.60} at $256 \times 256$ resolution within only 320 epochs. Furthermore, at $512 \times 512$ resolution, FrequencyBooster attains an FID of \textbf{1.69}, significantly outperforming existing pixel-space and latent-space generative models.

Related papers

• WaiT for the Signal: Simple Frequency-Aware Flow-Matching [60.56691023057048]
  We introduce WaiT, a Wavelet-aware image Transformer that decomposes generation into coarse and fine bands.<n>On ImageNet 512x512, WaiT achieves a pixel-space FID of 1.43, reducing sampling compute by up to 50%.<n>Our formulation outperforms even the strongest latent-space models on texture fidelity, and scales seamlessly to high-resolution OpenImages and to video generation.
  arXiv  Detail & Related papers  (2026-07-30T18:26:19Z)
• PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion [16.22743300700166]
  We introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space.<n> PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway"<n>Experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost.
  arXiv  Detail & Related papers  (2026-06-26T06:39:06Z)
• Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation [36.41177812868683]
  Latent diffusion models excel at generating high-quality images but lose the benefits of end-to-end modeling.<n>We propose Latent Forcing, a simple modification to existing architectures that achieves the efficiency of latent diffusion while operating on raw natural images.<n>Latent Forcing achieves a new state-of-the-art for diffusion transformer-based pixel generation at our compute scale.
  arXiv  Detail & Related papers  (2026-02-11T22:09:58Z)
• DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation [93.6273078684831]
  We propose a frequency-DeCoupled pixel diffusion framework to pursue a more efficient pixel diffusion paradigm.<n>With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance.<n>Experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet.
  arXiv  Detail & Related papers  (2025-11-24T17:59:06Z)
• DiP: Taming Diffusion Models in Pixel Space [91.51011771517683]
  Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction.<n>Co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details.
  arXiv  Detail & Related papers  (2025-11-24T06:55:49Z)
• HDW-SR: High-Frequency Guided Diffusion Model based on Wavelet Decomposition for Image Super-Resolution [4.388490927225987]
  We propose a High-Frequency Guided Diffusion Network based on Wavelet Decomposition (HDW-SR)<n>We perform diffusion only on the residual map, allowing the network to focus more effectively on high-frequency information restoration.<n> Experiments on both synthetic and real-world datasets demonstrate that HDW-SR achieves competitive super-resolution performance.
  arXiv  Detail & Related papers  (2025-11-17T09:25:26Z)
• ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration [75.0053551643052]
  We introduce ZipIR, a novel framework that enhances efficiency, scalability, and long-range modeling for high-res image restoration.<n>ZipIR employs a highly compressed latent representation that compresses image 32x, effectively reducing the number of spatial tokens.<n>ZipIR surpasses existing diffusion-based methods, offering unmatched speed and quality in restoring high-resolution images from severely degraded inputs.
  arXiv  Detail & Related papers  (2025-04-11T14:49:52Z)
• QMambaBSR: Burst Image Super-Resolution with Query State Space Model [55.56075874424194]
  Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames.<n>In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance.<n>We introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp)
  arXiv  Detail & Related papers  (2024-08-16T11:15:29Z)
• Frequency-Domain Refinement with Multiscale Diffusion for Super Resolution [19.327571569959062]
  We propose a novel Frequency Domain-guided multiscale Diffusion model (FDDiff)<n>FDDiff decomposes the high-frequency information complementing process into finer-grained steps.<n>It outperforms prior generative methods with higher-fidelity super-resolution results.
  arXiv  Detail & Related papers  (2024-05-16T11:58:52Z)
• ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models [126.35334860896373]
  We investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes. Existing works for higher-resolution generation, such as attention-based and joint-diffusion approaches, cannot well address these issues. We propose a simple yet effective re-dilation that can dynamically adjust the convolutional perception field during inference.
  arXiv  Detail & Related papers  (2023-10-11T17:52:39Z)
• Frequency Compensated Diffusion Model for Real-scene Dehazing [6.105813272271171]
  We consider a dehazing framework based on conditional diffusion models for improved generalization to real haze. The proposed dehazing diffusion model significantly outperforms state-of-the-art methods on real-world images.
  arXiv  Detail & Related papers  (2023-08-21T06:50:44Z)
• Improved Transformer for High-Resolution GANs [69.42469272015481]
  We introduce two key ingredients to Transformer to address this challenge. We show in the experiments that the proposed HiT achieves state-of-the-art FID scores of 31.87 and 2.95 on unconditional ImageNet $128 times 128$ and FFHQ $256 times 256$, respectively.
  arXiv  Detail & Related papers  (2021-06-14T17:39:49Z)
• Cascaded Diffusion Models for High Fidelity Image Generation [53.57766722279425]
  We show that cascaded diffusion models are capable of generating high fidelity images on the class-conditional ImageNet generation challenge. A cascaded diffusion model comprises a pipeline of multiple diffusion models that generate images of increasing resolution. We find that the sample quality of a cascading pipeline relies crucially on conditioning augmentation.
  arXiv  Detail & Related papers  (2021-05-30T17:14:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.