UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
Abstract Overview
UniGen-AR studies unified visual generation with a single model that produces image-valued outputs for more than 15 tasks across text-to-image generation, editing, restoration, and classical perception. The framework couples a multimodal language model encoder with a next-scale visual auto-regressive decoder, using a unified prompting interface and a single set of weights rather than task-specific heads. Reference images and instructions are encoded into a shared conditioning sequence, while the VAR decoder generates discrete visual tokens that are decoded back into images or dense prediction maps. The paper positions this design as an alternative to diffusion-based unified systems, aiming to preserve flexible multimodal conditioning while reducing inference cost.
Novelty
The paper presents what it describes as the first framework to extend next-scale visual auto-regressive modeling to full unified visual generation with MLLM-based conditioning. A distinctive element is the combination of an MLLM interface, prepended finest-scale reference tokens, and a unified autoregressive schedule that supports synthesis, editing, restoration, and perception within one image-output backbone.
Results
Empirically, UniGen-AR reports competitive performance across the evaluated task groups, with especially strong results relative to prior AR-based UVG systems on restoration and perception benchmarks. On the reported benchmarks, it improves over the Infinity (UVG) baseline on editing, depth, normals, low-light enhancement, deblurring, and deraining, while remaining competitive on text-to-image generation. For efficiency, the paper reports up to 19× lower inference latency than diffusion-based baselines overall, and in a controlled matched-setting comparison shows about 5× lower latency together with more than 10× lower FLOPs.
Key Points
- UniGen-AR unifies over 15 visual generation and image-output tasks in a single MLLM-conditioned VAR framework with no task-specific heads.
- The model shows broad empirical competence, particularly outperforming prior AR-based UVG baselines on restoration and perception tasks while staying competitive on text-to-image generation and editing.
- A central technical finding is that visual tokenizer design, especially VQ-VAE codebook size and hierarchy, materially affects the scalability and quality of VAR-based unified visual generation.