Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification

• URL: http://arxiv.org/abs/2606.18249v2
• Date: Wed, 17 Jun 2026 18:39:52 GMT
• Title: Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification
• Abstract summary: UniAR is a unified autoregressive framework where a single visual tokenizer serves as the key bridge between understanding and generation.<n>UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme.<n>A diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images.
• Score: 80.62512020268626
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re-encoding. UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.

Related papers

• Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling [22.020464703520148]
  We introduce DenseAR, a new generative paradigm that reformulates autoregressive image generation as coarse-to-fine next-dense-stride prediction.<n>Our key insight is that traversing a single-scale latent grid with progressively denser strides naturally captures the transition from global structure to fine detail.
  arXiv  Detail & Related papers  (2026-07-10T18:32:27Z)
• Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation [66.53544128707817]
  Cheers is a unified multimodal model that decouples patch-level details from semantic representations.<n> Cheers matches or surpasses advanced UMMs in both visual understanding and generation.
  arXiv  Detail & Related papers  (2026-03-13T08:55:27Z)
• UniCompress: Token Compression for Unified Vision-Language Understanding and Generation [62.943173382496276]
  Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text.<n>We propose a unified token compression algorithm that significantly reduces visual token count while preserving performance on both image understanding and generation tasks.
  arXiv  Detail & Related papers  (2026-03-11T21:27:15Z)
• ResTok: Learning Hierarchical Residuals in 1D Visual Tokenizers for Autoregressive Image Generation [64.84095852784714]
  Residual Tokenizer (ResTok) is a 1D visual tokenizer that builds hierarchical residuals for both image tokens and latent tokens.<n>We show that restoring hierarchical residual priors in visual tokenization significantly improves AR image generation, achieving a gFID of 2.34 on ImageNet-256 with only 9 sampling steps.
  arXiv  Detail & Related papers  (2026-01-07T14:09:18Z)
• Growing Visual Generative Capacity for Pre-Trained MLLMs [60.826355079902505]
  Bridge is a pure autoregressive unified MLLM that augments pre-trained visual understanding models with generative ability.<n>We propose a semantic-to-pixel discrete representation that integrates compact semantic tokens with fine-grained pixel tokens.
  arXiv  Detail & Related papers  (2025-10-02T00:40:02Z)
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer [90.72238747690972]
  We present Manzano, a simple and scalable unified framework for multimodal large language models.<n>A single vision encoder feeds two adapters that produce continuous embeddings for image-to-text understanding and discrete tokens for text-to-image generation.<n>A unified autoregressive LLM predicts high-level semantics in the form of text and image tokens, with an auxiliary diffusion decoder subsequently translating the image tokens into pixels.
  arXiv  Detail & Related papers  (2025-09-19T17:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.