DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation

• URL: http://arxiv.org/abs/2511.19365v1
• Date: Mon, 24 Nov 2025 17:59:06 GMT
• Title: DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation
• Abstract summary: We propose a frequency-DeCoupled pixel diffusion framework to pursue a more efficient pixel diffusion paradigm.<n>With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance.<n>Experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet.
• Score: 93.6273078684831
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pixel diffusion aims to generate images directly in pixel space in an end-to-end fashion. This approach avoids the limitations of VAE in the two-stage latent diffusion, offering higher model capacity. Existing pixel diffusion models suffer from slow training and inference, as they usually model both high-frequency signals and low-frequency semantics within a single diffusion transformer (DiT). To pursue a more efficient pixel diffusion paradigm, we propose the frequency-DeCoupled pixel diffusion framework. With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance from the DiT. This thus frees the DiT to specialize in modeling low-frequency semantics. In addition, we introduce a frequency-aware flow-matching loss that emphasizes visually salient frequencies while suppressing insignificant ones. Extensive experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet, closing the gap with latent diffusion methods. Furthermore, our pretrained text-to-image model achieves a leading overall score of 0.86 on GenEval in system-level comparison. Codes are publicly available at https://github.com/Zehong-Ma/DeCo.

Related papers

• A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples [53.8898272536272]
  Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels.<n>Recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures.<n>We show there is a cheaper, complementary strategy: textbfa frozen, pretrained pixel diffusion model can guide itself
  arXiv  Detail & Related papers  (2026-07-31T07:52:08Z)
• PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion [16.22743300700166]
  We introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space.<n> PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway"<n>Experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost.
  arXiv  Detail & Related papers  (2026-06-26T06:39:06Z)
• PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion [65.47126282928896]
  latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details.<n>We introduce PiD, a Pixel diffusion Decoder that reformulates latent decoding as conditional pixel diffusion.<n>By denoising directly in high-resolution pixel space, PiD synthesizes $4times$ and even $8times$ upscaled images with low latency.
  arXiv  Detail & Related papers  (2026-05-22T17:59:42Z)
• FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion [15.557373713923306]
  FrequencyBooster is a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead.<n>Our model achieves a state-of-the-art FID of textbf1.60 at $256 times 256$ resolution within only 320 epochs.<n>At $512 times 512$ resolution, FrequencyBooster attains an FID of textbf1.69, significantly outperforming existing pixel-space and latent-space generative models.
  arXiv  Detail & Related papers  (2026-05-18T02:25:07Z)
• PixelGen: Pixel Diffusion Beats Latent Diffusion with Perceptual Loss [47.868429337792314]
  We propose PixelGen, a simple pixel diffusion framework with perceptual supervision.<n>Instead of modeling the full image manifold, PixelGen introduces two complementary perceptual losses.<n>An LPIPS loss facilitates learning better local patterns, while a DINO-based perceptual loss strengthens global semantics.
  arXiv  Detail & Related papers  (2026-02-02T18:59:42Z)
• PixelDiT: Pixel Diffusion Transformers for Image Generation [48.456815413366535]
  PixelDiT is a single-stage, end-to-end model for Diffusion Transformers.<n>It eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space.<n>It achieves 1.61 FID on ImageNet 256x256, surpassing existing pixel generative models by a large margin.
  arXiv  Detail & Related papers  (2025-11-25T18:59:25Z)
• One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models [65.96186414865747]
  Text-to-Image (T2I) diffusion models face a trade-off between inference speed and image quality.<n>We introduce the first Time-independent Unified TiUE for the student model UNet architecture.<n>Using a one-pass scheme, TiUE shares encoder features across multiple decoder time steps, enabling parallel sampling.
  arXiv  Detail & Related papers  (2025-05-28T04:23:22Z)
• DDT: Decoupled Diffusion Transformer [51.84206763079382]
  Diffusion transformers encode noisy inputs to extract semantic component and decode higher frequency with identical modules.<n>textbfcolorddtDecoupled textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)
  arXiv  Detail & Related papers  (2025-04-08T07:17:45Z)
• Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion [34.70370851239368]
  We show that pixel-space models can be very competitive to latent models both in quality and efficiency.<n>We present a simple recipe for scaling end-to-end pixel-space diffusion models to high resolutions.
  arXiv  Detail & Related papers  (2024-10-25T06:20:06Z)
• StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [52.56469577812338]
  We introduce StreamDiffusion, a real-time diffusion pipeline for interactive image generation.<n>Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction.<n>We present a novel approach that transforms the original sequential denoising into the denoising process.
  arXiv  Detail & Related papers  (2023-12-19T18:18:33Z)
• One-step Diffusion with Distribution Matching Distillation [54.723565605974294]
  We introduce Distribution Matching Distillation (DMD), a procedure to transform a diffusion model into a one-step image generator. We enforce the one-step image generator match the diffusion model at distribution level, by minimizing an approximate KL divergence. Our method outperforms all published few-step diffusion approaches, reaching 2.62 FID on ImageNet 64x64 and 11.49 FID on zero-shot COCO-30k.
  arXiv  Detail & Related papers  (2023-11-30T18:59:20Z)
• Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image Synthesis [39.671396431940224]
  diffusion models have shown remarkable results in image synthesis by gradually removing noise and amplifying signals. We propose a novel generative process that synthesizes images in a coarse-to-fine manner. Experiments show that the proposed model outperforms the previous method in FID on LSUN bedroom and church datasets.
  arXiv  Detail & Related papers  (2022-07-16T15:00:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.