Emu3.5: Native Multimodal Models are World Learners

• URL: http://arxiv.org/abs/2510.26583v1
• Date: Thu, 30 Oct 2025 15:11:16 GMT
• Title: Emu3.5: Native Multimodal Models are World Learners
• Abstract summary: We introduce Emu3.5, a large-scale multimodal world model that natively predicts the next state across vision and language.<n>Emu3.5 is pre-trained end-to-end with a unified next-token prediction objective on a corpus of vision-language interleaved data.<n>It exhibits generalizable world-modeling abilities enabling consistent world exploration and open-world embodied manipulation.
• Score: 65.85558430499516
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Emu3.5, a large-scale multimodal world model that natively predicts the next state across vision and language. Emu3.5 is pre-trained end-to-end with a unified next-token prediction objective on a corpus of vision-language interleaved data containing over 10 trillion tokens, primarily derived from sequential frames and transcripts of internet videos. The model naturally accepts interleaved vision-language inputs and generates interleaved vision-language outputs. Emu3.5 is further post-trained with large-scale reinforcement learning to enhance multimodal reasoning and generation. To improve inference efficiency, we propose Discrete Diffusion Adaptation (DiDA), which converts token-by-token decoding into bidirectional parallel prediction, accelerating per-image inference by about 20x without sacrificing performance. Emu3.5 exhibits strong native multimodal capabilities, including long-horizon vision-language generation, any-to-image (X2I) generation, and complex text-rich image generation. It also exhibits generalizable world-modeling abilities, enabling spatiotemporally consistent world exploration and open-world embodied manipulation across diverse scenarios and tasks. For comparison, Emu3.5 achieves performance comparable to Gemini 2.5 Flash Image (Nano Banana) on image generation and editing tasks and demonstrates superior results on a suite of interleaved generation tasks. We open-source Emu3.5 at https://github.com/baaivision/Emu3.5 to support community research.

Related papers

• GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation [27.566010743301675]
  We propose a novel unified Driving World Models framework based on 3D Gaussian scene representation.<n>Our approach directly aligns textual information with the 3D scene by embedding rich linguistic features into each Gaussian primitive.<n>In addition, we design a novel task-aware language-guided sampling strategy that removes redundant 3D Gaussians and injects accurate and compact 3D tokens into LLM.
  arXiv  Detail & Related papers  (2025-12-29T03:40:05Z)
• Show-o2: Improved Native Unified Multimodal Models [57.34173415412808]
  Show-o2 is a native unified multimodal models that leverage autoregressive modeling and flow matching.<n>Built upon a 3D causal variational autoencoder space, unified visual representations are constructed through a dual-path of spatial (-temporal) fusion.
  arXiv  Detail & Related papers  (2025-06-18T15:39:15Z)
• ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding [16.95099884066268]
  ShapeLLM- Omni is a native 3D large language model capable of understanding and generating 3D assets and text in any sequence.<n>Building upon the 3D-aware discrete tokens, we innovatively construct a large-scale continuous training dataset named 3D-Alpaca.<n>Our work provides an effective attempt at extending multimodal models with basic 3D capabilities, which contributes to future research in 3D-native AI.
  arXiv  Detail & Related papers  (2025-06-02T16:40:50Z)
• The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation [53.837937703425794]
  LanDiff is a hybrid framework that synergizes the strengths of autoregressive language models and diffusion models.<n>Our architecture introduces three key innovations: (1) a semantic tokenizer that compresses 3D visual features into compact 1D representations through efficient semantic compression; (2) a language model that generates semantic tokens with high-level semantic relationships; and (3) a streaming diffusion model that refines coarse semantics into high-fidelity videos.
  arXiv  Detail & Related papers  (2025-03-06T16:53:14Z)
• Emu3: Next-Token Prediction is All You Need [45.142268281651035]
  We introduce Emu3, a new suite of state-of-the-art multimodal models trained solely with next-token prediction. Emu3 outperforms several well-established task-specific models in both generation and perception tasks. It is also capable of generating high-fidelity video via predicting the next token in a video sequence.
  arXiv  Detail & Related papers  (2024-09-27T16:06:11Z)
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation [72.94143731623117]
  Existing methods simply align 3D representations with single-view 2D images and coarse-grained parent category text. Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space. We propose a multi-view joint modality modeling approach, termed JM3D, to obtain a unified representation for point cloud, text, and image.
  arXiv  Detail & Related papers  (2023-08-06T01:11:40Z)
• Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation [72.74191015833397]
  We propose TransFusion, a multimodal transformer-based architecture. It exploits the representational power of language by summarizing the action context. Our model enables more efficient end-to-end learning.
  arXiv  Detail & Related papers  (2023-01-22T21:30:12Z)
• Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks [87.6494641931349]
  We introduce a general-purpose multimodal foundation model BEiT-3. It achieves state-of-the-art transfer performance on both vision and vision-language tasks.
  arXiv  Detail & Related papers  (2022-08-22T16:55:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.