Hyperspherical Latents Improve Continuous-Token Autoregressive Generation

• URL: http://arxiv.org/abs/2509.24335v1
• Date: Mon, 29 Sep 2025 06:34:24 GMT
• Title: Hyperspherical Latents Improve Continuous-Token Autoregressive Generation
• Abstract summary: Autoregressive (AR) models are promising for image generation, yet continuous-token AR variants often trail latent diffusion and masked-generation models.<n>We propose SphereAR to address the issue of heterogeneous variance in VAE latents, which is amplified during AR decoding.<n>Our theoretical analysis shows that hyperspherical constraint removes the scale component, thereby stabilizing AR decoding.
• Score: 13.664450684805535
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Autoregressive (AR) models are promising for image generation, yet continuous-token AR variants often trail latent diffusion and masked-generation models. The core issue is heterogeneous variance in VAE latents, which is amplified during AR decoding, especially under classifier-free guidance (CFG), and can cause variance collapse. We propose SphereAR to address this issue. Its core design is to constrain all AR inputs and outputs -- including after CFG -- to lie on a fixed-radius hypersphere (constant $\ell_2$ norm), leveraging hyperspherical VAEs. Our theoretical analysis shows that hyperspherical constraint removes the scale component (the primary cause of variance collapse), thereby stabilizing AR decoding. Empirically, on ImageNet generation, SphereAR-H (943M) sets a new state of the art for AR models, achieving FID 1.34. Even at smaller scales, SphereAR-L (479M) reaches FID 1.54 and SphereAR-B (208M) reaches 1.92, matching or surpassing much larger baselines such as MAR-H (943M, 1.55) and VAR-d30 (2B, 1.92). To our knowledge, this is the first time a pure next-token AR image generator with raster order surpasses diffusion and masked-generation models at comparable parameter scales.

Related papers

• DiffusionGemma Technical Report [72.10570774996219]
  We introduce DiffusionGemma, a language model that uses discrete diffusion to generate text at exceptionally high speed.<n>Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel.<n>Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget.
  arXiv  Detail & Related papers  (2026-07-31T16:11:46Z)
• Autoregressive Boltzmann Generators [81.5061620844556]
  Efficient sampling of molecular systems at thermodynamic equilibrium is a hallmark challenge in statistical physics.<n>Modern Boltzmann Generators predominantly rely on normalizing flows.<n>We propose Autoregressive Boltzmann Generators (ArBG) that departs from the flow-based BG paradigm.
  arXiv  Detail & Related papers  (2026-06-25T17:58:21Z)
• VARestorer: One-Step VAR Distillation for Real-World Image Super-Resolution [76.19751531910039]
  We propose VARestorer, a framework that transforms a pre-trained text-to-image VAR model into a one-step ISR model.<n>VARestorer achieves state-of-the-art performance with 72.32 MUSIQ and 0.7669 CLIPIQA on DIV2K dataset.
  arXiv  Detail & Related papers  (2026-04-23T09:09:36Z)
• MAR-GRPO: Stabilized GRPO for AR-diffusion Hybrid Image Generation [24.618644100413018]
  Reinforcement learning (RL) has been successfully applied to autoregressive (AR) and diffusion models.<n> extending RL to hybrid AR-diffusion frameworks remains challenging due to interleaved inference and noisy log-probability estimation.<n>In this work, we study masked autoregressive models (MAR) and show that the diffusion head plays a critical role in training dynamics.
  arXiv  Detail & Related papers  (2026-04-08T11:30:35Z)
• RAE-AR: Taming Autoregressive Models with Representation Autoencoders [61.73674018219353]
  We introduce token simplification via distribution normalization to ease modeling difficulty and improve convergence.<n>We enhance prediction by incorporating Gaussian noise injection during training to mitigate exposure bias.<n>This work paves the way for a more unified architecture across visual understanding and generative modeling.
  arXiv  Detail & Related papers  (2026-04-02T02:39:28Z)
• TiDAR: Think in Diffusion, Talk in Autoregression [59.94106070312094]
  TiDAR is a sequence-level hybrid architecture that drafts tokens (Thinking) in Diffusion and samples final outputs (Talking) AutoRegressively.<n> TiDAR is the first architecture to close the quality gap with AR models while delivering 4.71x to 5.91x more tokens per second.
  arXiv  Detail & Related papers  (2025-11-12T02:59:33Z)
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation [62.14510717860079]
  We propose a Synergistic Diffusion-Autoregression paradigm that unifies the training efficiency of autoregressive models with the parallel inference capability of diffusion.<n>SDAR performs a lightweight paradigm conversion that transforms a well-trained autoregressive (AR) model into a blockwise diffusion model through brief, data-efficient adaptation.<n>Building on this insight, SDAR achieves efficient AR-to-diffusion conversion with minimal cost, preserving AR-level performance while enabling parallel generation.
  arXiv  Detail & Related papers  (2025-10-07T17:29:28Z)
• $\f{D^3}$QE: Learning Discrete Distribution Discrepancy-aware Quantization Error for Autoregressive-Generated Image Detection [85.9202830503973]
  Visual autoregressive (AR) models generate images through discrete token prediction.<n>We propose to leverage Discrete Distribution Discrepancy-aware Quantization Error (D$3$QE) for autoregressive-generated image detection.
  arXiv  Detail & Related papers  (2025-10-07T13:02:27Z)
• REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularization [130.46612643194973]
  reAR is a simple training strategy introducing a token-wise regularization objective.<n>On ImageNet, it reduces gFID from 3.02 to 1.86 and improves IS to 316.9 using a standardization-based tokenizer.<n>When applied to advanced tokenizers, it achieves a gFID of 1.42 with only 177M parameters, matching the performance with larger state-of-the-art diffusion models (675M)
  arXiv  Detail & Related papers  (2025-10-06T02:48:13Z)
• Scale-Wise VAR is Secretly Discrete Diffusion [48.994983608261286]
  Next scale prediction Visual Autoregressive Generation ( VAR) has recently demonstrated remarkable performance, even surpassing diffusion-based models.<n>In this work, we revisit VAR and uncover a theoretical insight: when equipped with a Markovian attention mask, VAR is mathematically equivalent to a discrete diffusion.<n>We show how one can directly import the advantages of diffusion such as iterative refinement and reduce architectural inefficiencies into VAR, yielding faster convergence, lower inference cost, and improved zero-shot reconstruction.
  arXiv  Detail & Related papers  (2025-09-26T17:58:04Z)
• Diffusion Beats Autoregressive in Data-Constrained Settings [50.56893491038853]
  Autoregressive (AR) models have long dominated the landscape of large language models, driving progress across a wide range of tasks.<n>Recently, diffusion-based language models have emerged as a promising alternative, though their advantages over AR models remain underexplored.<n>We systematically study masked diffusion models in data-constrained settings where training involves repeated passes over limited data.<n>Our results suggest that when data, not compute, is the bottleneck, diffusion models offer a compelling alternative to the standard AR paradigm.
  arXiv  Detail & Related papers  (2025-07-21T17:59:57Z)
• DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer [32.64616770377737]
  DC-AR is a novel masked autoregressive (AR) text-to-image generation framework.<n>It delivers superior image generation quality with exceptional computational efficiency.
  arXiv  Detail & Related papers  (2025-07-07T12:45:23Z)
• Multi-scale Image Super Resolution with a Single Auto-Regressive Model [40.77470215283583]
  We tackle Image Super Resolution (ISR) using recent advances in Visual Auto-Regressive ( VAR) modeling.<n>To the best of our knowledge, this is the first time a quantizer is trained to force semantically consistent residuals at different scales.<n>Our model can denoise the LR image and super-resolve at half and full target upscale factors in a single forward pass.
  arXiv  Detail & Related papers  (2025-06-05T13:02:23Z)
• One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation [60.54811860967658]
  FluxSR is a novel one-step diffusion Real-ISR based on flow matching models.<n>First, we introduce Flow Trajectory Distillation (FTD) to distill a multi-step flow matching model into a one-step Real-ISR.<n>Second, to improve image realism and address high-frequency artifact issues in generated images, we propose TV-LPIPS as a perceptual loss.
  arXiv  Detail & Related papers  (2025-02-04T04:11:29Z)
• Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching [12.985270202599814]
  Autoregressive (AR) models have achieved state-of-the-art performance in text and image generation but suffer from slow generation due to the token-by-token process.<n>We ask an ambitious question: can a pre-trained AR model be adapted to generate outputs in just one or two steps?<n>We propose Distilled Decoding (DD), which uses flow matching to create a deterministic mapping from Gaussian distribution to the output distribution of the pre-trained AR model.
  arXiv  Detail & Related papers  (2024-12-22T20:21:54Z)
• M-VAR: Decoupled Scale-wise Autoregressive Modeling for High-Quality Image Generation [39.97174784206976]
  We show that this scale-wise autoregressive framework can be effectively decoupled into textitintra-scale modeling We apply linear-complexity mechanisms like Mamba to substantially reduce computational overhead. Experiments demonstrate that our method outperforms existing models in both image quality and generation speed.
  arXiv  Detail & Related papers  (2024-11-15T18:54:42Z)
• HART: Efficient Visual Generation with Hybrid Autoregressive Transformer [33.97880303341509]
  We introduce Hybrid Autoregressive Transformer (HART), an autoregressive (AR) visual generation model capable of directly generating 1024x1024 images. Our approach improves reconstruction FID from 2.11 to 0.30 on MJHQ-30K, leading to a 31% generation FID improvement from 7.85 to 5.38. HART also outperforms state-of-the-art diffusion models in both FID and CLIP score, with 4.5-7.7x higher throughput and 6.9-13.4x lower MACs.
  arXiv  Detail & Related papers  (2024-10-14T17:59:42Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Unsupervised Single Image Super-resolution Under Complex Noise [60.566471567837574]
  This paper proposes a model-based unsupervised SISR method to deal with the general SISR task with unknown degradations. The proposed method can evidently surpass the current state of the art (SotA) method (about 1dB PSNR) not only with a slighter model (0.34M vs. 2.40M) but also faster speed.
  arXiv  Detail & Related papers  (2021-07-02T11:55:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.