OmniGen-AR: AutoRegressive Any-to-Image Generation
Abstract Overview
OmniGen-AR is presented as a unified autoregressive framework for any-to-image generation that encodes text and diverse visual conditions into discrete tokens within a single model. Using a shared visual tokenizer for images, depth maps, segmentation masks, and other visual inputs, plus a text tokenizer for prompts, the system supports text-to-image, segmentation-to-image, depth-to-image, image editing, frame prediction, and text-to-video generation. The paper identifies information leakage between condition and target tokens as a key issue in conditional autoregressive generation and introduces Disentangled Causal Attention (DCA) as a training-time regularization strategy to separate condition and content attention paths. Experiments across multiple benchmarks indicate that the approach is broadly effective, with particularly strong results in text-to-image and text-to-video generation while remaining competitive on several other controlled generation tasks.
Novelty
The main novelty is a unified autoregressive any-to-image formulation that handles multiple conditioning modalities through a shared discrete token space rather than separate modality-specific encoders or adapters. The paper also introduces Disentangled Causal Attention, a training-time masking scheme designed to reduce shortcut learning and information leakage from condition tokens to generated content tokens.
Results
On the reported benchmarks, the 1.5B OmniGen-AR model reaches 0.63 on GenEval for text-to-image generation and 80.02 on VBench for text-to-video generation. The model also shows competitive image editing performance (0.23 CLIP text similarity, 0.84 CLIP image similarity) and lower zero-shot frame-prediction FVD than VideoPoet, while spatially conditioned generation is competitive but not the top result in the reported table. Ablation results show that applying DCA with a 10% probability improves editing-related conditioning performance relative to using standard causal attention alone.
Key Points
- OmniGen-AR unifies text, spatial controls, and visual context within a single autoregressive model by tokenizing them into a shared discrete representation.
- The proposed Disentangled Causal Attention is a training-time regularizer that aims to reduce information leakage and improve instruction-following in conditional generation tasks.
- Empirical results are strongest on GenEval and VBench, while performance on editing, frame prediction, and spatially conditioned generation is generally competitive across tasks.