Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis

• URL: http://arxiv.org/abs/2507.01756v2
• Date: Tue, 22 Jul 2025 02:25:49 GMT
• Title: Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis
• Authors: Peng Zheng, Junke Wang, Yi Chang, Yizhou Yu, Rui Ma, Zuxuan Wu,
• Abstract summary: DisCon is a novel framework that reinterprets discrete tokens as conditional signals rather than generation targets.<n>DisCon achieves a gFID score of 1.38 on ImageNet 256$times $256 generation, outperforming state-of-the-art autoregressive approaches by a clear margin.
• Score: 79.98107530577576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in large language models (LLMs) have spurred interests in encoding images as discrete tokens and leveraging autoregressive (AR) frameworks for visual generation. However, the quantization process in AR-based visual generation models inherently introduces information loss that degrades image fidelity. To mitigate this limitation, recent studies have explored to autoregressively predict continuous tokens. Unlike discrete tokens that reside in a structured and bounded space, continuous representations exist in an unbounded, high-dimensional space, making density estimation more challenging and increasing the risk of generating out-of-distribution artifacts. Based on the above findings, this work introduces DisCon (Discrete-Conditioned Continuous Autoregressive Model), a novel framework that reinterprets discrete tokens as conditional signals rather than generation targets. By modeling the conditional probability of continuous representations conditioned on discrete tokens, DisCon circumvents the optimization challenges of continuous token modeling while avoiding the information loss caused by quantization. DisCon achieves a gFID score of 1.38 on ImageNet 256$\times$256 generation, outperforming state-of-the-art autoregressive approaches by a clear margin. Project page: https://pengzheng0707.github.io/DisCon.

Related papers

• Autoregressive Image Generation with Masked Bit Modeling [34.36577356251466]
  Bit AutoRegressive modeling (BAR) is a scalable framework that supports arbitrary codebook sizes.<n>BAR achieves a new state-of-the-art gFID of 0.99 on ImageNet-256, outperforming leading methods across both continuous and discrete paradigms.
  arXiv  Detail & Related papers  (2026-02-09T18:59:58Z)
• ResTok: Learning Hierarchical Residuals in 1D Visual Tokenizers for Autoregressive Image Generation [64.84095852784714]
  Residual Tokenizer (ResTok) is a 1D visual tokenizer that builds hierarchical residuals for both image tokens and latent tokens.<n>We show that restoring hierarchical residual priors in visual tokenization significantly improves AR image generation, achieving a gFID of 2.34 on ImageNet-256 with only 9 sampling steps.
  arXiv  Detail & Related papers  (2026-01-07T14:09:18Z)
• SFTok: Bridging the Performance Gap in Discrete Tokenizers [72.9996757048065]
  We propose textbfSFTok, a discrete tokenizer that incorporates a multi-step iterative mechanism for precise reconstruction.<n>At a high compression rate of only 64 tokens per image, SFTok achieves state-of-the-art reconstruction quality on ImageNet.
  arXiv  Detail & Related papers  (2025-12-18T18:59:04Z)
• ARSS: Taming Decoder-only Autoregressive Visual Generation for View Synthesis From Single View [11.346049532150127]
  textbfARSS is a framework that generates novel views from a single image conditioned on a camera trajectory.<n>Our method performs comparably to, or better than, state-of-the-art view synthesis approaches based on diffusion models.
  arXiv  Detail & Related papers  (2025-09-27T00:03:09Z)
• NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale [101.57871281101747]
  NextStep-1 achieves state-of-the-art performance for autoregressive models in text-to-image generation tasks.<n>Our method shows strong performance in image editing, highlighting the power and versatility of our unified approach.
  arXiv  Detail & Related papers  (2025-08-14T14:54:22Z)
• Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models [92.18057318458528]
  Token-Shuffle is a novel method that reduces the number of image tokens in Transformer.<n>Our strategy requires no additional pretrained text-encoder and enables MLLMs to support extremely high-resolution image synthesis.<n>In GenAI-benchmark, our 2.7B model achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen by 0.18 and diffusion models LDM by 0.15.
  arXiv  Detail & Related papers  (2025-04-24T17:59:56Z)
• Fast Autoregressive Models for Continuous Latent Generation [49.079819389916764]
  Autoregressive models have demonstrated remarkable success in sequential data generation, particularly in NLP.<n>Recent work, the masked autoregressive model (MAR) bypasses quantization by modeling per-token distributions in continuous spaces using a diffusion head.<n>We propose Fast AutoRegressive model (FAR), a novel framework that replaces MAR's diffusion head with a lightweight shortcut head.
  arXiv  Detail & Related papers  (2025-04-24T13:57:08Z)
• D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens [80.75893450536577]
  We propose D2C, a novel two-stage method to enhance model generation capacity.<n>In the first stage, the discrete-valued tokens representing coarse-grained image features are sampled by employing a small discrete-valued generator.<n>In the second stage, the continuous-valued tokens representing fine-grained image features are learned conditioned on the discrete token sequence.
  arXiv  Detail & Related papers  (2025-03-21T13:58:49Z)
• Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation [63.89280381800457]
  We propose TokenBridge, which maintains the strong representation capacity of continuous tokens while preserving the modeling simplicity of discrete tokens.<n>We introduce a dimension-wise quantization strategy that independently discretizes each feature dimension, paired with a lightweight autoregressive prediction mechanism.<n>Our approach achieves reconstruction and generation quality on par with continuous methods while using standard categorical prediction.
  arXiv  Detail & Related papers  (2025-03-20T17:59:59Z)
• Improving Autoregressive Image Generation through Coarse-to-Fine Token Prediction [4.900334213807624]
  We show how to enjoy the benefits of large codebooks without making autoregressive modeling more difficult.<n>Our framework consists of two stages: (1) an autoregressive model that sequentially predicts coarse labels for each token in the sequence, and (2) an auxiliary model that simultaneously predicts fine-grained labels for all tokens conditioned on their coarse labels.
  arXiv  Detail & Related papers  (2025-03-20T14:41:29Z)
• NFIG: Autoregressive Image Generation with Next-Frequency Prediction [50.69346038028673]
  We present textbfNext-textbfFrequency textbfImage textbfGeneration (textbfNFIG), a novel framework that decomposes the image generation process into multiple frequency-guided stages.<n>Our approach first generates low-frequency components to establish global structure with fewer tokens, then progressively adds higher-frequency details, following the natural spectral hierarchy of images.
  arXiv  Detail & Related papers  (2025-03-10T08:59:10Z)
• Frequency Autoregressive Image Generation with Continuous Tokens [31.833852108014312]
  We introduce the frequency progressive autoregressive (textbfFAR) paradigm and instantiate FAR with the continuous tokenizer.<n>We demonstrate the efficacy of FAR through comprehensive experiments on the ImageNet dataset.
  arXiv  Detail & Related papers  (2025-03-07T10:34:04Z)
• Continuous Speculative Decoding for Autoregressive Image Generation [33.05392461723613]
  Continuous-valued Autoregressive (AR) image generation models have demonstrated notable superiority over their discrete-token counterparts. speculative decoding has proven effective in accelerating Large Language Models (LLMs) This work generalizes the speculative decoding algorithm from discrete tokens to continuous space.
  arXiv  Detail & Related papers  (2024-11-18T09:19:15Z)
• MMAR: Towards Lossless Multi-Modal Auto-Regressive Probabilistic Modeling [64.09238330331195]
  We propose a novel Multi-Modal Auto-Regressive (MMAR) probabilistic modeling framework.<n>Unlike discretization line of method, MMAR takes in continuous-valued image tokens to avoid information loss in an efficient way.<n>We also propose a theoretically proven technique that addresses the numerical stability issue and a training strategy that balances the generation and understanding task goals.
  arXiv  Detail & Related papers  (2024-10-14T17:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.