A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples

• URL: http://arxiv.org/abs/2607.29122v1
• Date: Fri, 31 Jul 2026 07:52:08 GMT
• Title: A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples
• Abstract summary: Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels.<n>Recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures.<n>We show there is a cheaper, complementary strategy: textbfa frozen, pretrained pixel diffusion model can guide itself
• Score: 53.8898272536272
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels. This is challenging because a single model must capture both global structure and local texture in the same high-dimensional space. While recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures, these advances typically require training a new model from scratch. We show there is a cheaper, complementary strategy: \textbf{a frozen, pretrained pixel diffusion model can guide itself}. Our key observation is that intermediate layers of a pretrained pixel diffusion transformer can be decoded into coarse predictions that capture the main low-frequency structure, while the final layers progressively refine local, high-frequency details. We therefore attach a lightweight prediction head to an intermediate layer, keep the backbone frozen, and use the discrepancy between the intermediate and final predictions as a self-guidance direction during sampling. To train this head, we further find that real images are not necessary. Instead, model-generated samples suffice and even outperform real images for training the head, especially in enhancing the high-frequency components that pixel diffusion tends to underfit. Across multiple pixel diffusion models on ImageNet, our \textbf{Synthetic Self-Guidance (SSG)} consistently improves generation while adapter training requires less than 1$\%$ of full-model training compute: it reduces FID by over 50$\%$ across the evaluated JiT variants without classifier-free guidance (CFG) and further improves strong baselines with CFG, e.g., JiT-H/16 from 1.86 to 1.67 and PixelREPA-H/16 from 1.81 to 1.59. Our code is available at https://github.com/zfu006/SSG.

Related papers

• Representation Forcing for Bottleneck-Free Unified Multimodal Models [76.99907273945493]
  Unified multimodal models (UMMs) aim to handle perception and generation in a single model.<n>Existing UMMs still rely on a frozen, separately pretrained VAE for image generation, imposing a structural bottleneck.<n>In this paper, we propose Representation Forcing (RF), a technique that closes this gap by making representation prediction a native capability of the model.
  arXiv  Detail & Related papers  (2026-05-29T17:59:55Z)
• PixelDiT: Pixel Diffusion Transformers for Image Generation [48.456815413366535]
  PixelDiT is a single-stage, end-to-end model for Diffusion Transformers.<n>It eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space.<n>It achieves 1.61 FID on ImageNet 256x256, surpassing existing pixel generative models by a large margin.
  arXiv  Detail & Related papers  (2025-11-25T18:59:25Z)
• DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation [93.6273078684831]
  We propose a frequency-DeCoupled pixel diffusion framework to pursue a more efficient pixel diffusion paradigm.<n>With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance.<n>Experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet.
  arXiv  Detail & Related papers  (2025-11-24T17:59:06Z)
• Advancing End-to-End Pixel Space Generative Modeling via Self-supervised Pre-training [23.632047555553324]
  We introduce a novel two-stage training framework for pixel-space diffusion and consistency models.<n>Our training framework demonstrates strong empirical performance on ImageNet dataset.<n>To the best of our knowledge, this marks the first successful training of a consistency model directly on high-resolution images.
  arXiv  Detail & Related papers  (2025-10-14T14:41:16Z)
• Context-guided Responsible Data Augmentation with Diffusion Models [29.41191005466334]
  We propose a text-to-image (T2I) data augmentation method, named DiffCoRe-Mix, that computes a set of generative counterparts for a training sample.<n>To preserve key semantic axes, we also filter out undesired generative samples in our augmentation process.<n>We extensively evaluate our technique on ImageNet-1K,Tiny ImageNet-200, CIFAR-100, Flowers102, CUB-Birds, Stanford Cars, and Caltech datasets.
  arXiv  Detail & Related papers  (2025-03-12T00:12:27Z)
• David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training [8.352666876052616]
  We propose Diff-Instruct* (DI*), a data-efficient post-training approach for one-step text-to-image generative models.<n>Our method frames alignment as online reinforcement learning from human feedback.<n>Our 2.6B emphDI*-SDXL-1step model outperforms the 50-step 12B FLUX-dev model.
  arXiv  Detail & Related papers  (2024-10-28T10:26:19Z)
• Fast constrained sampling in pre-trained diffusion models [80.99262780028015]
  We propose an algorithm that enables fast, high-quality generation under arbitrary constraints.<n>Our approach produces results that rival or surpass the state-of-the-art training-free inference methods.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models [49.80911683739506]
  We focus on the task of image segmentation, which is traditionally solved by training models on closed-vocabulary datasets.<n>We leverage different and relatively small-sized, open-source foundation models for zero-shot open-vocabulary segmentation.<n>Our approach (dubbed FreeSeg-Diff), which does not rely on any training, outperforms many training-based approaches on both Pascal VOC and COCO datasets.
  arXiv  Detail & Related papers  (2024-03-29T10:38:25Z)
• ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation [15.057394130958583]
  ElasticDiffusion is a training-free decoding method that enables pretrained text-to-image diffusion models to generate images with various sizes.<n>Our experiments and qualitative results show superior image coherence quality across aspect ratios compared to MultiDiffusion and the standard decoding strategy of Stable Diffusion.
  arXiv  Detail & Related papers  (2023-11-30T18:58:17Z)
• Masked Diffusion Models Are Fast Distribution Learners [32.485235866596064]
  Diffusion models are commonly trained to learn all fine-grained visual information from scratch. We show that it suffices to train a strong diffusion model by first pre-training the model to learn some primer distribution. Then the pre-trained model can be fine-tuned for various generation tasks efficiently.
  arXiv  Detail & Related papers  (2023-06-20T08:02:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.