A Simple Baseline for Unifying Understanding, Generation, and Editing via Vanilla Next-token Prediction

• URL: http://arxiv.org/abs/2603.04980v1
• Date: Thu, 05 Mar 2026 09:17:44 GMT
• Title: A Simple Baseline for Unifying Understanding, Generation, and Editing via Vanilla Next-token Prediction
• Abstract summary: We introduce Wallaroo, a simple autoregressive baseline that leverages next-token prediction to unify multi-modal understanding, image generation, and editing.<n>Wallaroo supports multi-resolution image input and output, as well as bilingual support for both Chinese and English.
• Score: 18.67411100452757
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this work, we introduce Wallaroo, a simple autoregressive baseline that leverages next-token prediction to unify multi-modal understanding, image generation, and editing at the same time. Moreover, Wallaroo supports multi-resolution image input and output, as well as bilingual support for both Chinese and English. We decouple the visual encoding into separate pathways and apply a four-stage training strategy to reshape the model's capabilities. Experiments are conducted on various benchmarks where Wallaroo produces competitive performance or exceeds other unified models, suggesting the great potential of autoregressive models in unifying multi-modality understanding and generation. Our code is available at https://github.com/JiePKU/Wallaroo.

Related papers

• NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation [66.92488610008519]
  NextFlow is a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image discrete tokens.<n>By leveraging a unified vision representation within a unified autoregressive architecture, NextFlow activates multimodal understanding and generation capabilities.<n>NextFlow achieves state-of-the-art performance among unified models and rivals specialized diffusion baselines in visual quality.
  arXiv  Detail & Related papers  (2026-01-05T15:27:04Z)
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer [90.72238747690972]
  We present Manzano, a simple and scalable unified framework for multimodal large language models.<n>A single vision encoder feeds two adapters that produce continuous embeddings for image-to-text understanding and discrete tokens for text-to-image generation.<n>A unified autoregressive LLM predicts high-level semantics in the form of text and image tokens, with an auxiliary diffusion decoder subsequently translating the image tokens into pixels.
  arXiv  Detail & Related papers  (2025-09-19T17:58:00Z)
• Show-o2: Improved Native Unified Multimodal Models [57.34173415412808]
  Show-o2 is a native unified multimodal models that leverage autoregressive modeling and flow matching.<n>Built upon a 3D causal variational autoencoder space, unified visual representations are constructed through a dual-path of spatial (-temporal) fusion.
  arXiv  Detail & Related papers  (2025-06-18T15:39:15Z)
• Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models [92.18057318458528]
  Token-Shuffle is a novel method that reduces the number of image tokens in Transformer.<n>Our strategy requires no additional pretrained text-encoder and enables MLLMs to support extremely high-resolution image synthesis.<n>In GenAI-benchmark, our 2.7B model achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen by 0.18 and diffusion models LDM by 0.15.
  arXiv  Detail & Related papers  (2025-04-24T17:59:56Z)
• Emu3: Next-Token Prediction is All You Need [45.142268281651035]
  We introduce Emu3, a new suite of state-of-the-art multimodal models trained solely with next-token prediction. Emu3 outperforms several well-established task-specific models in both generation and perception tasks. It is also capable of generating high-fidelity video via predicting the next token in a video sequence.
  arXiv  Detail & Related papers  (2024-09-27T16:06:11Z)
• Show-o: One Single Transformer to Unify Multimodal Understanding and Generation [71.24909962718128]
  We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation.<n>Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities.
  arXiv  Detail & Related papers  (2024-08-22T16:32:32Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.