M$^\text{4}$World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming

• URL: http://arxiv.org/abs/2607.14005v1
• Date: Wed, 15 Jul 2026 16:36:33 GMT
• Title: M$^\text{4}$World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming
• Abstract summary: M$text4$World is a Multi-view and Multimodal generative driving world model.<n>Fine-grained object manipulation is realized through a flexible conditioning interface.<n>Stable minute-long streaming is achieved through a multi-stage training framework.
• Score: 26.912855667055307
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Driving-world generation has emerged as a core capability for scalable autonomous-driving simulation, yet existing methods remain limited in object-level controllability and long-horizon stability. We present M$^\text{4}$World, a Multi-view and Multimodal generative driving world model that synthesizes future surround-view video streams and synchronized LiDAR scans while supporting interactive object Manipulation and stable Minute-long streaming. Fine-grained object manipulation is realized through a flexible conditioning interface that supports explicit control over both the spatial layout and visual appearance of individual objects. Stable minute-long streaming, on the other hand, is achieved through a multi-stage training framework that enables online causal generation in only four denoising steps while maintaining coherent world dynamics throughout extended rollouts. Building on these components, we introduce an efficient few-clip post-training as well as a suite of visual reference-conditioned generation models, preserving general generation ability while allowing rare-case customization for long-tail controllability. To assess controllability beyond realism, we further introduce an automated VLM-based judging pipeline that evaluates scene-level condition adherence, view-wise object controllability, and cross-view object consistency. Comprehensive experiments show that M$^\text{4}$World consistently delivers high generation quality, precise controllability, and stable minute-long streaming. Together with downstream long-tail augmentation and scene editing, these results demonstrate the potential of M$^\text{4}$World for controllable, scalable driving simulation.

Related papers

• EchoWM: Open and Enterable Omnimodal World Models [71.49148067556887]
  EchoWM is an omnimodal world model for enterable generative media.<n>It responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech.
  arXiv  Detail & Related papers  (2026-08-24T12:39:59Z)
• Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control [31.020641426071293]
  We introduce Worldscape-MoE, a Mixture-of-Experts world model for scalable heterogeneous action control.<n>Our key observation is that different controls specify different interfaces to the same underlying world.<n>Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy.
  arXiv  Detail & Related papers  (2026-07-04T17:45:38Z)
• WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning [4.427336047705734]
  Vision-Language-Action (VLA) models have recently demonstrated strong generalization in robotic manipulation.<n>This work introduces a unified framework WOLF-VLA that integrates whole-body optimal-control (OC) motion synthesis with large-scale multi-modal dataset.<n>We construct a comprehensive dataset of dynamically feasible humanoid trajectories across six locomotion-related task families.<n>We train a VLA model using the collected joint trajectories, ego-centric visual observations and natural language instruction, yielding a policy that exhibits strong reasoning and robustness to initial-condition variability.
  arXiv  Detail & Related papers  (2026-06-24T08:59:59Z)
• DisCo: World Models with Discrete Camera Motion Control [79.86256515640231]
  We propose DisCo, a controllable video world model that conditions generation on a compact set of discrete action primitives to improve action separability.<n>DisCo achieves significantly more reliable action following while preserving visual quality.
  arXiv  Detail & Related papers  (2026-06-06T03:50:45Z)
• MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
  MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
  arXiv  Detail & Related papers  (2026-06-01T18:20:20Z)
• X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving [45.260296714057766]
  We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space.<n>X-World supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control.<n>Experiments show that X-World achieves high-quality multi-view video generation with strong view consistency across cameras.
  arXiv  Detail & Related papers  (2026-03-20T14:23:32Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control [122.65089441381741]
  We present GEM, a Generalizable Ego-vision Multimodal world model.<n>It predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories.<n>Our dataset is comprised of 4000+ hours of multimodal data across domains like autonomous driving, egocentric human activities, and drone flights.
  arXiv  Detail & Related papers  (2024-12-15T14:21:19Z)
• Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention [61.3281618482513]
  We present CogDriving, a novel network designed for synthesizing high-quality multi-view driving videos.<n>CogDriving leverages a Diffusion Transformer architecture with holistic-4D attention modules, enabling simultaneous associations across the dimensions.<n>CogDriving demonstrates strong performance on the nuScenes validation set, achieving an FVD score of 37.8, highlighting its ability to generate realistic driving videos.
  arXiv  Detail & Related papers  (2024-12-04T18:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.