MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data

• URL: http://arxiv.org/abs/2606.02753v1
• Date: Mon, 01 Jun 2026 18:20:20 GMT
• Title: MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data
• Abstract summary: MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
• Score: 125.43597497646444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video world models are a foundational generative technology for embodied AI and the Metaverse, yet existing approaches are inherently limited to a single agent observing from a single perspective. Extending these models to multi-agent settings introduces two critical challenges: data scarcity (coordinated multi-view recordings are prohibitively expensive to collect for general open-domain scenarios) and world state alignment (independently generated video streams cannot ensure that shared physical environments and events evolve consistently across views). To address these challenges, we propose MetaWorld, a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos. First, we introduce Monocular World-State Unrolling (MWSU) to explicitly decompose monocular footage into the camera operator's ego-motion and the visible subject's spatial trajectory. This camera-trajectory decomposition naturally extracts synchronized multi-agent motion data within a shared 3D space, completely bypassing the need for multi-camera setups. Second, for precise visual control, we develop the Subject-Aware World Generator to enable appearance-driven simulation conditioned on per-agent identity images. Finally, to ensure both views are grounded in the identical physical reality, we propose World-State Alignment, a per-frame inter-branch cross-attention mechanism inserted at every transformer layer of the video DiT. By jointly synchronizing the denoising process, WSA enforces both static geometric consistency and dynamic motion consistency, encouraging that the shared 3D environment and physical events remain well-aligned across both egocentric views. Extensive experiments demonstrate that MetaWorld achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.

Related papers

• EchoWM: Open and Enterable Omnimodal World Models [71.49148067556887]
  EchoWM is an omnimodal world model for enterable generative media.<n>It responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech.
  arXiv  Detail & Related papers  (2026-08-24T12:39:59Z)
• Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers [50.3000377325823]
  We present WorldWeaver, a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers.<n>We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text.<n>Experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.
  arXiv  Detail & Related papers  (2026-07-23T17:59:58Z)
• WorldOlympiad: Can Your World Model Survive a Triathlon? [39.69359039523777]
  We introduce WorldOlympiad, a benchmark for diagnosing video-based world models across physical faithfulness, geometric consistency, and interaction fidelity.<n>WorldOlympiad decomposes world-model evaluation into three complementary dimensions.
  arXiv  Detail & Related papers  (2026-06-09T17:24:36Z)
• Prisma-World: Camera-Controllable Multi-Agent Video World Model [67.72842238020192]
  We introduce Prisma-World, a camera-controllable multi-agent world model.<n>It formulates multi-agent generation as a joint geometry-aware denoising process for cross-view consistency.<n>Experiments show that a single Prisma-World model can generate high-fidelity multi-agent videos with flexible agent numbers.
  arXiv  Detail & Related papers  (2026-06-08T13:59:50Z)
• Embody4D: A Generalist 4D World Model for Embodied AI [13.488596291534243]
  Embody4D is a dedicated video-to-video world model for embodied scenarios.<n>It is capable of arbitrary novel views from a monocular video.<n>It achieves state-of-the-art performance, serving as a robust world model.
  arXiv  Detail & Related papers  (2026-05-03T09:39:44Z)
• VideoWeaver: Multimodal Multi-View Video-to-Video Transfer for Embodied Agents [17.66237759970927]
  VideoWeaver is the first multimodal multi-view V2V translation framework.<n>We train views at distinct diffusion timesteps, enabling the model to learn both joint and conditional view distributions.<n>Experiments show superior or similar performance to the state-of-the-art on the single-view translation benchmarks.
  arXiv  Detail & Related papers  (2026-03-26T13:14:13Z)
• MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model [83.12400850233034]
  Human-Object Interaction (HOI) video reenactment with realistic motion remains a frontier in digital human creation.<n>We propose MVHOI, a two-stage HOI video reenactment framework that bridges multi-view reference conditions and video foundation models.<n>Our framework shows superior performance in generating long-duration HOI videos with intricate object manipulations.
  arXiv  Detail & Related papers  (2026-03-16T00:43:38Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• IC-World: In-Context Generation for Shared World Modeling [61.69655562995357]
  Video-based world models have recently garnered increasing attention for their ability to synthesize diverse and dynamic visual environments.<n>In this paper, we focus on shared world modeling, where a model generates multiple videos from a set of input images, each representing the same underlying world in different camera poses.<n>We propose IC-World, a novel generation framework, enabling parallel generation for all input images.
  arXiv  Detail & Related papers  (2025-12-01T16:52:02Z)
• Towards High-Consistency Embodied World Model with Multi-View Trajectory Videos [24.111891848073288]
  Embodied world models aim to predict and interact with the physical world through visual observations and actions.<n>MTV-World introduces Multi-view Trajectory-Video control for precise visuomotor prediction.<n>MTV-World achieves precise control execution and accurate physical interaction modeling in complex dual-arm scenarios.
  arXiv  Detail & Related papers  (2025-11-17T02:17:04Z)
• ERMV: Editing 4D Robotic Multi-view images to enhance embodied agents [14.75400720374728]
  ERMV ( Robotic Multi-View 4D data framework) efficiently edits an entire multi-view sequence based on single-frame editing and robot state conditions.<n>Emerged data significantly boosts robustness and guidance of models in both simulated and real-world environments.
  arXiv  Detail & Related papers  (2025-07-23T12:41:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.