HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer

• URL: http://arxiv.org/abs/2605.11061v1
• Date: Mon, 11 May 2026 17:59:09 GMT
• Title: HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer
• Abstract summary: We present HiDream-O1-Image, a unified generative foundation model via pixel-space Diffusion Transformer.<n>By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs.<n>Experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization.
• Score: 104.09730595701468
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The evolution of visual generative models has long been constrained by fragmented architectures relying on disjoint text encoders and external VAEs. In this report, we present HiDream-O1-Image, a natively unified generative foundation model via pixel-space Diffusion Transformer, that pioneers a paradigm shift from modular architectures to an end-to-end in-context visual generation engine. By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs within an Unified Transformer (UiT) architecture. This native encoding paradigm eliminates the need for separate VAEs or disjoint pre-trained text encoders, allowing the model to treat diverse generation and editing tasks as a consistent in-context reasoning process. Extensive experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization. Notably, with only 8B parameters, HiDream-O1-Image (8B) achieves performance parity with or even surpasses established state-of-the-art models with significantly larger parameters (e.g., 27B Qwen-Image). Crucially, to validate the immense scalability of this paradigm, we successfully scale the architecture up to over 200B parameters. Experimental results demonstrate that this massive-scale version HiDream-O1-Image-Pro (200B+) unlocks unprecedented generative capabilities and superior performance, establishing new state-of-the-art benchmarks. Ultimately, HiDream-O1-Image highlights the immense potential of natively unified architectures and charts a highly scalable path toward next-generation multimodal AI.

Related papers

• Imagine Before You Draw: Visual Prompt Engineering for Image Generation [67.81347924426714]
  We propose Visual Prompt Engineering (VPE), which can be seamlessly integrated into internal frameworks.<n>We validate VPE across class-conditional generation, text-to-image generation, and image editing.<n>Results show that VPE can accelerate convergence, raise quality ceilings, and through internal integration, achieve substantially better editing preservation.
  arXiv  Detail & Related papers  (2026-06-03T05:01:36Z)
• DuoGen: Towards General Purpose Interleaved Multimodal Generation [65.13479486098419]
  DuoGen is a general-purpose interleaved generation framework that addresses data curation, architecture design, and evaluation.<n>We build a large-scale, high-quality instruction-tuning dataset by combining multimodal conversations rewritten from curated raw websites.<n>A two-stage decoupled strategy first instruction-tunes the MLLM, then aligns DiT with it using curated interleaved image-text sequences.
  arXiv  Detail & Related papers  (2026-01-31T04:35:15Z)
• UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation [54.38636515750502]
  We propose UniLayDiff: a Unified Diffusion Transformer for content-aware layout generation tasks.<n>We employ Multi-Modal Diffusion Transformer framework to capture the complex interplay between the background image, layout elements, and diverse constraints.<n>Experiments demonstrate that UniLayDiff achieves state-of-the-art performance across from unconditional to various conditional generation tasks.
  arXiv  Detail & Related papers  (2025-12-09T18:38:44Z)
• Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generation [54.588082888166504]
  We present Mogao, a unified framework that enables interleaved multi-modal generation through a causal approach.<n>Mogoo integrates a set of key technical improvements in architecture design, including a deep-fusion design, dual vision encoders, interleaved rotary position embeddings, and multi-modal classifier-free guidance.<n>Experiments show that Mogao achieves state-of-the-art performance in multi-modal understanding and text-to-image generation, but also excels in producing high-quality, coherent interleaved outputs.
  arXiv  Detail & Related papers  (2025-05-08T17:58:57Z)
• OminiControl: Minimal and Universal Control for Diffusion Transformer [68.3243031301164]
  We present OminiControl, a novel approach that rethinks how image conditions are integrated into Diffusion Transformer (DiT) architectures.<n>OminiControl addresses these limitations through three key innovations.
  arXiv  Detail & Related papers  (2024-11-22T17:55:15Z)
• ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer [40.32254040909614]
  We propose ACE, an All-round Creator and Editor, for visual generation tasks. We first introduce a unified condition format termed Long-context Condition Unit (LCU) We then propose a novel Transformer-based diffusion model that uses LCU as input, aiming for joint training across various generation and editing tasks.
  arXiv  Detail & Related papers  (2024-09-30T17:56:27Z)
• Modality-Specialized Synergizers for Interleaved Vision-Language Generalists [45.800383191637785]
  Vision-Language Generalists (VLGs) capable of understanding and generating both text and images.<n>One primary limitation lies in applying a unified architecture and the same set of parameters to simultaneously model discrete text tokens and continuous image features.<n>Recent works attempt to tackle this fundamental problem by introducing modality-aware expert models.<n>We introduce MODALITY-SPECIALIZED SYNERGIZERS (MOSS), a novel design that efficiently optimize existing unified architectures of VLGs.
  arXiv  Detail & Related papers  (2024-07-04T03:28:22Z)
• GIT: A Generative Image-to-text Transformer for Vision and Language [138.91581326369837]
  We train a Generative Image-to-text Transformer, GIT, to unify vision-language tasks such as image/video captioning and question answering. Our model surpasses the human performance for the first time on TextCaps (138.2 vs. 125.5 in CIDEr)
  arXiv  Detail & Related papers  (2022-05-27T17:03:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.