STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation

• URL: http://arxiv.org/abs/2605.08029v1
• Date: Fri, 08 May 2026 17:14:43 GMT
• Title: STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation
• Abstract summary: We present autoregressive normalizing flows, sharing the same causal mask, KV-cache mechanism, and left-to-right structure as LLMs.<n>We demonstrate strong performance across image generation and multimodal understanding benchmarks, validating autoregressive flows as a viable foundation for unified multimodal modeling.
• Score: 56.964292897565024
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep generative models have advanced rapidly across text and vision, motivating unified multimodal systems that can understand, reason over, and generate interleaved text-image sequences. Most existing approaches combine autoregressive language modeling with diffusion-based image generators, inheriting a structural mismatch between causal text generation and iterative visual denoising. We observe that autoregressive normalizing flows are autoregressive Transformers--sharing the same causal mask, KV-cache mechanism, and left-to-right structure as LLMs--making them the most natural paradigm for true unified multimodal generation. We present STARFlow2, built on the Pretzel architecture that vertically interleaves a pretrained VLM stream with a TarFlow stream via residual skip connections, both operating under the same causal mask. Combined with a deep-shallow flow design and a unified FAE latent space, STARFlow2 enables cache-friendly interleaved generation where both text and visual outputs directly enter the KV-cache without re-encoding. Experiments demonstrate strong performance across image generation and multimodal understanding benchmarks, validating autoregressive flows as a viable foundation for unified multimodal modeling.

Related papers

• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation [66.92488610008519]
  NextFlow is a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image discrete tokens.<n>By leveraging a unified vision representation within a unified autoregressive architecture, NextFlow activates multimodal understanding and generation capabilities.<n>NextFlow achieves state-of-the-art performance among unified models and rivals specialized diffusion baselines in visual quality.
  arXiv  Detail & Related papers  (2026-01-05T15:27:04Z)
• OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows [59.052955667723985]
  We present OneFlow, the first non-autoregressive multimodal model that enables variable-length and concurrent mixed-modal generation.<n>Unlike autoregressive models that enforce rigid causal ordering between text and image generation, OneFlow combines an insertion-based Edit Flow for discrete text tokens with Flow Matching for image latents.
  arXiv  Detail & Related papers  (2025-10-03T20:40:30Z)
• Query-Kontext: An Unified Multimodal Model for Image Generation and Editing [53.765351127477224]
  Unified Multimodal Models (UMMs) have demonstrated remarkable performance in text-to-image generation (T2I) and editing (TI2I)<n>We introduce Query-Kontext, a novel approach that bridges the VLM and diffusion model via a multimodal kontext'' composed of semantic cues and coarse-grained image conditions encoded from multimodal inputs.<n> Experiments show that our approach matches strong unified baselines and even outperforms task-specific state-of-the-art methods in several cases.
  arXiv  Detail & Related papers  (2025-09-30T17:59:46Z)
• Show-o2: Improved Native Unified Multimodal Models [57.34173415412808]
  Show-o2 is a native unified multimodal models that leverage autoregressive modeling and flow matching.<n>Built upon a 3D causal variational autoencoder space, unified visual representations are constructed through a dual-path of spatial (-temporal) fusion.
  arXiv  Detail & Related papers  (2025-06-18T15:39:15Z)
• V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation [20.436493280226337]
  We propose V2Flow, a novel tokenizer that produces discrete visual tokens capable of high-fidelity reconstruction.<n>V2Flow enables autoregressive visual generation on top of existing language models.
  arXiv  Detail & Related papers  (2025-03-10T16:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.