PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion

• URL: http://arxiv.org/abs/2606.27760v1
• Date: Fri, 26 Jun 2026 06:39:06 GMT
• Title: PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion
• Abstract summary: We introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space.<n> PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway"<n>Experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost.
• Score: 16.22743300700166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end pixel-space diffusion models bypass the lossy compression of Latent Diffusion Models (LDMs) but struggle to jointly model low-frequency semantics and high-frequency signals in high-dimensional space. Existing works heavily rely on complex pixel decoders to alleviate this issue. In this paper, we challenge this trend by revealing that these decoders primarily compensate for the optimization difficulties inherent to velocity prediction ($v$-prediction). Under the clean data paradigm ($x$-prediction), they are redundant. Motivated by this insight, we advocate for simplicity over complexity and introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space. PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway" that directly routes uncorrupted high-frequency spatial details from shallow to deep layers. To further enable the backbone to focus exclusively on modeling low-frequency semantics, we introduce a constant-channel spatial down-sampling mechanism as a natural low-pass filter, which compresses deep features into a compact, low-frequency semantic manifold. Extensive experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost. On ImageNet 256$\times$256 and 512$\times$512, PixelU achieves FID of 1.63 and 1.92 respectively, surpassing recent pixel-space methods and establishing a simple yet powerful new paradigm for end-to-end diffusion models.

Related papers

• Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling [49.11379121512358]
  We propose a unified generative framework that leverages pre-trained Diffusion Transformer priors to achieve high perceptual quality at extremely lows.<n>Our method achieves state-of-the-art perceptual fidelity with rich spatial details and robust temporal consistency.
  arXiv  Detail & Related papers  (2026-07-21T06:36:32Z)
• PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion [65.47126282928896]
  latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details.<n>We introduce PiD, a Pixel diffusion Decoder that reformulates latent decoding as conditional pixel diffusion.<n>By denoising directly in high-resolution pixel space, PiD synthesizes $4times$ and even $8times$ upscaled images with low latency.
  arXiv  Detail & Related papers  (2026-05-22T17:59:42Z)
• FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion [15.557373713923306]
  FrequencyBooster is a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead.<n>Our model achieves a state-of-the-art FID of textbf1.60 at $256 times 256$ resolution within only 320 epochs.<n>At $512 times 512$ resolution, FrequencyBooster attains an FID of textbf1.69, significantly outperforming existing pixel-space and latent-space generative models.
  arXiv  Detail & Related papers  (2026-05-18T02:25:07Z)
• PixelGen: Pixel Diffusion Beats Latent Diffusion with Perceptual Loss [47.868429337792314]
  We propose PixelGen, a simple pixel diffusion framework with perceptual supervision.<n>Instead of modeling the full image manifold, PixelGen introduces two complementary perceptual losses.<n>An LPIPS loss facilitates learning better local patterns, while a DINO-based perceptual loss strengthens global semantics.
  arXiv  Detail & Related papers  (2026-02-02T18:59:42Z)
• PixelDiT: Pixel Diffusion Transformers for Image Generation [48.456815413366535]
  PixelDiT is a single-stage, end-to-end model for Diffusion Transformers.<n>It eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space.<n>It achieves 1.61 FID on ImageNet 256x256, surpassing existing pixel generative models by a large margin.
  arXiv  Detail & Related papers  (2025-11-25T18:59:25Z)
• DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation [93.6273078684831]
  We propose a frequency-DeCoupled pixel diffusion framework to pursue a more efficient pixel diffusion paradigm.<n>With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance.<n>Experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet.
  arXiv  Detail & Related papers  (2025-11-24T17:59:06Z)
• DiP: Taming Diffusion Models in Pixel Space [91.51011771517683]
  Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction.<n>Co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details.
  arXiv  Detail & Related papers  (2025-11-24T06:55:49Z)
• One Small Step in Latent, One Giant Leap for Pixels: Fast Latent Upscale Adapter for Your Diffusion Models [45.92038137978053]
  We present the Latent Upscaler Adapter (LUA), a lightweight module that performs super-resolution directly on the generator's latent code.<n>LUA integrates as a drop-in component, requiring no modifications to the base model or additional diffusion stages.<n>A shared Swin-style backbone with scale-specific pixel-shuffle heads supports 2x and 4x factors and remains compatible with image-space SR baselines.
  arXiv  Detail & Related papers  (2025-11-13T18:54:18Z)
• DDT: Decoupled Diffusion Transformer [51.84206763079382]
  Diffusion transformers encode noisy inputs to extract semantic component and decode higher frequency with identical modules.<n>textbfcolorddtDecoupled textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)
  arXiv  Detail & Related papers  (2025-04-08T07:17:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.