Semantic-Aware Prefix Learning for Token-Efficient Image Generation

• URL: http://arxiv.org/abs/2603.25249v1
• Date: Thu, 26 Mar 2026 09:52:51 GMT
• Title: Semantic-Aware Prefix Learning for Token-Efficient Image Generation
• Abstract summary: We propose a SeMantic-Aware Prefix tokenizer that injects class-level semantic conditions into a query-based 1D tokenization framework.<n>To make semantics indispensable during training, SMAP introduces a tail token dropping strategy, which forces semantic conditions and early latent prefixes to bear increasing responsibility.<n>Experiments on ImageNet show that SMAP consistently improves reconstruction quality across discrete and continuous tokenization settings.
• Score: 15.50406566147261
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual tokenizers play a central role in latent image generation by bridging high-dimensional images and tractable generative modeling. However, most existing tokenizers are still trained with reconstruction-dominated objectives, which often yield latent representations that are only weakly grounded in high-level semantics. Recent approaches improve semantic alignment, but typically treat semantic signals as auxiliary regularization rather than making them functionally necessary for representation learning. We propose SMAP, a SeMantic-Aware Prefix tokenizer that injects class-level semantic conditions into a query-based 1D tokenization framework. To make semantics indispensable during training, SMAP introduces a tail token dropping strategy, which forces semantic conditions and early latent prefixes to bear increasing responsibility under progressively reduced token budgets. To verify that the resulting latent space is useful for generation rather than reconstruction alone, we further introduce CARD, a hybrid Causal AutoRegressive--Diffusion generator. Extensive experiments on ImageNet show that SMAP consistently improves reconstruction quality across discrete and continuous tokenization settings, and that its semantically grounded latent space yields strong downstream generation performance under compact token budgets.

Related papers

• Where Does Generative Difficulty Reside? An Empirical Study of Target Representations [62.54876287800644]
  The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface.<n>We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model.<n>Our results show that compression, reconstruction fidelity, token dimensionality, and visible semantic clustering do not individually predict generative behavior.
  arXiv  Detail & Related papers  (2026-08-01T12:32:31Z)
• Self-Corrected Image Generation with Explainable Latent Rewards [55.29175717238288]
  We propose xLARD, a self-correcting framework that guides generation through Explainable LAtent RewarDs.<n>xLARD introduces a lightweight corrector that refines latent representations based on structured feedback from model-generated references.<n> Experiments show that xLARD improves semantic alignment and visual fidelity while maintaining generative priors.
  arXiv  Detail & Related papers  (2026-03-26T02:59:35Z)
• Semantic One-Dimensional Tokenizer for Image Reconstruction and Generation [11.568334063059638]
  SemTok is a semantic one-dimensional tokenizer that compresses 2D images into 1D discretetext tokens with high-level semantics.<n> SemTok sets a new state-of-the-art in image reconstruction, achieving superior fidelity with a remarkably compact token representation.<n>Building on SemTok, we construct a masked autoregressive generation framework, which yields notable improvements in downstream image generation tasks.
  arXiv  Detail & Related papers  (2026-03-17T11:01:08Z)
• Learning Sparse Visual Representations via Spatial-Semantic Factorization [37.169502692169196]
  Self-supervised learning (SSL) faces a fundamental conflict between semantic understanding and image reconstruction.<n>We introduce STELLAR, a framework that factorizes visual features into a low-rank product of semantic concepts and their spatial distributions.<n>We demonstrate that as few as 16 sparse tokens under this factorized form are sufficient to simultaneously support high-quality reconstruction (2.60 FID) and match the semantic performance of dense backbones (79.10% ImageNet accuracy)
  arXiv  Detail & Related papers  (2026-02-02T10:12:17Z)
• ResTok: Learning Hierarchical Residuals in 1D Visual Tokenizers for Autoregressive Image Generation [64.84095852784714]
  Residual Tokenizer (ResTok) is a 1D visual tokenizer that builds hierarchical residuals for both image tokens and latent tokens.<n>We show that restoring hierarchical residual priors in visual tokenization significantly improves AR image generation, achieving a gFID of 2.34 on ImageNet-256 with only 9 sampling steps.
  arXiv  Detail & Related papers  (2026-01-07T14:09:18Z)
• SFTok: Bridging the Performance Gap in Discrete Tokenizers [72.9996757048065]
  We propose textbfSFTok, a discrete tokenizer that incorporates a multi-step iterative mechanism for precise reconstruction.<n>At a high compression rate of only 64 tokens per image, SFTok achieves state-of-the-art reconstruction quality on ImageNet.
  arXiv  Detail & Related papers  (2025-12-18T18:59:04Z)
• RecTok: Reconstruction Distillation along Rectified Flow [85.51292475005151]
  We propose RecTok, which overcomes the limitations of high-dimensional visual tokenizers through two key innovations.<n>Our method distills the semantic information in VFMs into the forward flow trajectories in flow matching.<n>Our RecTok achieves superior image reconstruction, generation quality, and discriminative performance.
  arXiv  Detail & Related papers  (2025-12-15T15:14:20Z)
• Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer [50.69959748410398]
  We introduce MingTok, a new family of visual tokenizers with a continuous latent space for unified autoregressive generation and understanding.<n>MingTok adopts a three-stage sequential architecture involving low-level encoding, semantic expansion, and visual reconstruction.<n>Built on top of it, Ming-UniVision eliminates the need for task-specific visual representations, and unifies diverse vision-language tasks under a single autoregrsssive prediction paradigm.
  arXiv  Detail & Related papers  (2025-10-08T02:50:14Z)
• Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation [85.82112629564942]
  We propose TokenBridge, which maintains the strong representation capacity of continuous tokens while preserving the modeling simplicity of discrete tokens.<n>We introduce a dimension-wise quantization strategy that independently discretizes each feature dimension, paired with a lightweight autoregressive prediction mechanism.<n>Our approach achieves reconstruction and generation quality on par with continuous methods while using standard categorical prediction.
  arXiv  Detail & Related papers  (2025-03-20T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.