PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion

• URL: http://arxiv.org/abs/2605.23902v1
• Date: Fri, 22 May 2026 17:59:42 GMT
• Title: PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion
• Abstract summary: latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details.<n>We introduce PiD, a Pixel diffusion Decoder that reformulates latent decoding as conditional pixel diffusion.<n>By denoising directly in high-resolution pixel space, PiD synthesizes $4times$ and even $8times$ upscaled images with low latency.
• Score: 65.47126282928896
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most practical high-resolution text-to-image systems, including latent diffusion and autoregressive models, perform generation in a compact latent space, and a decoder maps the generated latents back to pixels. Yet the latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details, and becomes increasingly costly at megapixel scale. This drawback calls for a more expressive and efficient decoding paradigm. Motivated by recent progress in scalable pixel-space diffusion, we introduce PiD, a Pixel diffusion Decoder that reformulates latent decoding as conditional pixel diffusion, unifying decoding and upsampling into one generative module. By denoising directly in high-resolution pixel space, PiD synthesizes $4\times$ and even $8\times$ upscaled images with low latency. For latent conditioning, a lightweight sigma-aware adapter injects noise-corrupted latents into the pixel diffusion backbone, enabling PiD to decode partially denoised latents and terminate the latent diffusion process early. To further improve efficiency, we distill the model using DMD2, reducing inference to just 4 steps. PiD applies to both conventional VAE latents and semantic latents (e.g., SigLIP, DINOv2) used in recent RAE-based models. PiD decodes latents of $512 \times 512$ images into $2048 \times 2048$ pixels in under 1 second with 13 GB peak memory on a consumer RTX 5090, and as fast as 210 ms on a GB200 GPU, about $6\times$ faster than cascaded diffusion-based super-resolution pipelines with better visual fidelity.

Related papers

• PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion [16.22743300700166]
  We introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space.<n> PixelU abandons auxiliary decoders in favor of zero-cost skip connections, which provide an "information highway"<n>Experiments demonstrate that this decoupling of frequencies could outperform the strong baseline (JiT-G) with only about 1/3 of its computation cost.
  arXiv  Detail & Related papers  (2026-06-26T06:39:06Z)
• Laminating Representation Autoencoders for Efficient Diffusion [18.989001805139573]
  Recent work has shown that diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents.<n>We introduce FlatDINO, a variational autoencoder that compresses this representation into a one-dimensional sequence of just 32 continuous tokens.
  arXiv  Detail & Related papers  (2026-02-04T18:57:33Z)
• PixelGen: Pixel Diffusion Beats Latent Diffusion with Perceptual Loss [47.868429337792314]
  We propose PixelGen, a simple pixel diffusion framework with perceptual supervision.<n>Instead of modeling the full image manifold, PixelGen introduces two complementary perceptual losses.<n>An LPIPS loss facilitates learning better local patterns, while a DINO-based perceptual loss strengthens global semantics.
  arXiv  Detail & Related papers  (2026-02-02T18:59:42Z)
• PixelDiT: Pixel Diffusion Transformers for Image Generation [48.456815413366535]
  PixelDiT is a single-stage, end-to-end model for Diffusion Transformers.<n>It eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space.<n>It achieves 1.61 FID on ImageNet 256x256, surpassing existing pixel generative models by a large margin.
  arXiv  Detail & Related papers  (2025-11-25T18:59:25Z)
• DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation [93.6273078684831]
  We propose a frequency-DeCoupled pixel diffusion framework to pursue a more efficient pixel diffusion paradigm.<n>With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance.<n>Experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet.
  arXiv  Detail & Related papers  (2025-11-24T17:59:06Z)
• One Small Step in Latent, One Giant Leap for Pixels: Fast Latent Upscale Adapter for Your Diffusion Models [45.92038137978053]
  We present the Latent Upscaler Adapter (LUA), a lightweight module that performs super-resolution directly on the generator's latent code.<n>LUA integrates as a drop-in component, requiring no modifications to the base model or additional diffusion stages.<n>A shared Swin-style backbone with scale-specific pixel-shuffle heads supports 2x and 4x factors and remains compatible with image-space SR baselines.
  arXiv  Detail & Related papers  (2025-11-13T18:54:18Z)
• StableCodec: Taming One-Step Diffusion for Extreme Image Compression [19.69733852050049]
  Diffusion-based image compression has shown remarkable potential for achieving ultra-low coding (less than 0.05 bits per pixel) with high realism.<n>Current approaches require a large number of denoising steps at the decoder to generate realistic results under extreme constraints.<n>We introduce StableCodec, which enables one-step diffusion for high-fidelity and high-realism extreme image compression.
  arXiv  Detail & Related papers  (2025-06-27T07:39:21Z)
• DDT: Decoupled Diffusion Transformer [51.84206763079382]
  Diffusion transformers encode noisy inputs to extract semantic component and decode higher frequency with identical modules.<n>textbfcolorddtDecoupled textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)
  arXiv  Detail & Related papers  (2025-04-08T07:17:45Z)
• PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher [55.22994720855957]
  PaGoDA is a novel pipeline that reduces the training costs through three stages: training diffusion on downsampled data, distilling the pretrained diffusion, and progressive super-resolution. With the proposed pipeline, PaGoDA achieves a $64times$ reduced cost in training its diffusion model on 8x downsampled data. PaGoDA's pipeline can be applied directly in the latent space, adding compression alongside the pre-trained autoencoder in Latent Diffusion Models.
  arXiv  Detail & Related papers  (2024-05-23T17:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.