OmniGen2: Exploration to Advanced Multimodal Generation

• URL: http://arxiv.org/abs/2506.18871v2
• Date: Wed, 25 Jun 2025 17:54:25 GMT
• Title: OmniGen2: Exploration to Advanced Multimodal Generation
• Abstract summary: We introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generation tasks.<n>Unlike OmniGen v1, OmniGen2 features two distinct decoding pathways for text and image modalities, utilizing unshared parameters and a decoupled image tokenizer.
• Score: 62.58494500322778
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generation tasks, including text-to-image, image editing, and in-context generation. Unlike OmniGen v1, OmniGen2 features two distinct decoding pathways for text and image modalities, utilizing unshared parameters and a decoupled image tokenizer. This design enables OmniGen2 to build upon existing multimodal understanding models without the need to re-adapt VAE inputs, thereby preserving the original text generation capabilities. To facilitate the training of OmniGen2, we developed comprehensive data construction pipelines, encompassing image editing and in-context generation data. Additionally, we introduce a reflection mechanism tailored for image generation tasks and curate a dedicated reflection dataset based on OmniGen2. Despite its relatively modest parameter size, OmniGen2 achieves competitive results on multiple task benchmarks, including text-to-image and image editing. To further evaluate in-context generation, also referred to as subject-driven tasks, we introduce a new benchmark named OmniContext. OmniGen2 achieves state-of-the-art performance among open-source models in terms of consistency. We will release our models, training code, datasets, and data construction pipeline to support future research in this field. Project Page: https://vectorspacelab.github.io/OmniGen2; GitHub Link: https://github.com/VectorSpaceLab/OmniGen2

Related papers

• OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning [81.93748829204145]
  We propose OmniWeaving, an omni-level video generation model with powerful multimodal composition and reasoning-informed capabilities.<n>We introduce IntelligentVBench, the first comprehensive benchmark designed to rigorously assess next-level intelligent unified video generation.<n>Experiments demonstrate that OmniWeaving achieves SoTA performance among open-source unified models.
  arXiv  Detail & Related papers  (2026-03-25T16:08:18Z)
• DuoGen: Towards General Purpose Interleaved Multimodal Generation [65.13479486098419]
  DuoGen is a general-purpose interleaved generation framework that addresses data curation, architecture design, and evaluation.<n>We build a large-scale, high-quality instruction-tuning dataset by combining multimodal conversations rewritten from curated raw websites.<n>A two-stage decoupled strategy first instruction-tunes the MLLM, then aligns DiT with it using curated interleaved image-text sequences.
  arXiv  Detail & Related papers  (2026-01-31T04:35:15Z)
• OmniMoGen: Unifying Human Motion Generation via Learning from Interleaved Text-Motion Instructions [50.40879516807197]
  We propose a unified framework that enables versatile motion generation through interleaved text-motion instructions.<n>Built upon a concise RVQ-VAE and transformer architecture, OmniMoGen supports end-to-end instruction-driven motion generation.<n>Experiments show that OmniMoGen achieves state-of-the-art performance on text-to-motion, motion editing, and AnyContext.
  arXiv  Detail & Related papers  (2025-12-22T08:55:23Z)
• Unlocking Aha Moments via Reinforcement Learning: Advancing Collaborative Visual Comprehension and Generation [85.22602924467603]
  We propose to enable the collaborative co-evolution of visual comprehension and generation.<n>We introduce a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT.<n>We unlock the Aha moment in visual generation, advancing MLLMs from text-to-image tasks to unified image generation.
  arXiv  Detail & Related papers  (2025-06-02T09:39:28Z)
• Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space [9.327655601475605]
  We propose Nexus-Gen, a novel architecture that unifies image understanding, generation, and editing tasks in a shared image embedding space.<n>To mitigate the severe error accumulation during autoregressive embedding prediction, we propose a novel prefilled autoregression strategy.<n>Nexus-Gen achieves state-of-the-art performance on the evaluation benchmarks spanning image understanding, generation and editing tasks.
  arXiv  Detail & Related papers  (2025-04-30T06:30:48Z)
• ImageGen-CoT: Enhancing Text-to-Image In-context Learning with Chain-of-Thought Reasoning [89.19449553099747]
  We study the problem of Text-to-Image In-Context Learning (T2I-ICL)<n>We propose a framework that incorporates a thought process called ImageGen-CoT prior to image generation.<n>We fine-tune MLLMs using this dataset to enhance their contextual reasoning capabilities.
  arXiv  Detail & Related papers  (2025-03-25T03:18:46Z)
• OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models [36.0400717590138]
  We present OmniMamba, the first linear-architecture-based multimodal generation model.<n>It generates both text and images through a unified next-token prediction paradigm.<n>It achieves competitive performance with JanusFlow while surpassing Show-o across benchmarks.
  arXiv  Detail & Related papers  (2025-03-11T17:59:46Z)
• OmniGen: Unified Image Generation [25.75041469368185]
  We present OmniGen, a new diffusion model for unified image generation. It supports various downstream tasks, such as image editing, subject-driven generation, and visual-conditional generation. The architecture of OmniGen is highly simplified, eliminating the need for additional plugins.
  arXiv  Detail & Related papers  (2024-09-17T16:42:46Z)
• OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation [95.29102596532854]
  Tokenizer serves as a translator to map the intricate visual data into a compact latent space. This paper presents OmniTokenizer, a transformer-based tokenizer for joint image and video tokenization.
  arXiv  Detail & Related papers  (2024-06-13T17:59:26Z)
• OmniControlNet: Dual-stage Integration for Conditional Image Generation [61.1432268643639]
  We provide a two-way integration for the widely adopted ControlNet by integrating external condition generation algorithms into a single dense prediction method. Our proposed OmniControlNet consolidates 1) the condition generation by a single multi-tasking dense prediction algorithm under the task embedding guidance and 2) the image generation process for different conditioning types under the textual embedding guidance.
  arXiv  Detail & Related papers  (2024-06-09T18:03:47Z)
• Instruct-Imagen: Image Generation with Multi-modal Instruction [90.04481955523514]
  instruct-imagen is a model that tackles heterogeneous image generation tasks and generalizes across unseen tasks. We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision. Human evaluation on various image generation datasets reveals that instruct-imagen matches or surpasses prior task-specific models in-domain.
  arXiv  Detail & Related papers  (2024-01-03T19:31:58Z)
• Multimodal Story Generation on Plural Images [8.293936347234126]
  We propose to use images as input of the text generation model, called StoryGen. The model demonstrates the ability to generate meaningful paragraphs of text containing the extracted features from the input images.
  arXiv  Detail & Related papers  (2020-01-16T03:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.