Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers

• URL: http://arxiv.org/abs/2607.21594v1
• Date: Thu, 23 Jul 2026 17:59:58 GMT
• Title: Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers
• Abstract summary: We present WorldWeaver, a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers.<n>We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text.<n>Experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.
• Score: 50.3000377325823
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings. We present WorldWeaver (W^2), a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text. We further improve the architecture with a Mixture-of-Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.

Related papers

• Population-Scalable Multi-Agent World Modeling [27.087757709788974]
  Khora is a scalable multi-agent world model that supports inference-time expansion to arbitrary numbers of agents without retraining.<n>Our framework decouples world-state evolution from visual rendering and introduces a population-agnostic rendering mechanism for incorporating other agent information.
  arXiv  Detail & Related papers  (2026-08-09T09:26:15Z)
• Prisma-World: Camera-Controllable Multi-Agent Video World Model [67.72842238020192]
  We introduce Prisma-World, a camera-controllable multi-agent world model.<n>It formulates multi-agent generation as a joint geometry-aware denoising process for cross-view consistency.<n>Experiments show that a single Prisma-World model can generate high-fidelity multi-agent videos with flexible agent numbers.
  arXiv  Detail & Related papers  (2026-06-08T13:59:50Z)
• MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
  MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
  arXiv  Detail & Related papers  (2026-06-01T18:20:20Z)
• Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players [87.87842088508553]
  We present our generative multi-agent world model for interactive simulation.<n>Our model improves video fidelity, action controllability, and inter-agent consistency over slot-based and dense-attention baselines.
  arXiv  Detail & Related papers  (2026-05-27T17:59:31Z)
• MultiWorld: Scalable Multi-Agent Multi-View Video World Models [36.970254855945484]
  We present textbfMultiWorld, a unified framework for multi-agent multi-view world modeling.<n>We introduce the Multi-Agent Condition Module to achieve precise multi-agent controllability.<n>Experiments on multi-player game environments and multi-robot manipulation tasks demonstrate that MultiWorld outperforms baselines in video fidelity, action-following ability, and multi-view consistency.
  arXiv  Detail & Related papers  (2026-04-20T17:52:15Z)
• ActionParty: Multi-Subject Action Binding in Generative Video Games [117.52562594944679]
  ActionParty is a controllable multi-subject world model for generative video games.<n>We demonstrate the first video world model capable of controlling up to seven players simultaneously across 46 diverse environments.
  arXiv  Detail & Related papers  (2026-04-02T17:59:58Z)
• ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modeling [16.27470663877285]
  This paper presents ShareVerse, a video generation framework enabling multi-agent shared world modeling.<n>A dataset for large-scale interactive world modeling is built on the CARLA simulation platform.<n>We propose a spatial concatenation strategy for four-view videos of independent agents to model a broader environment.<n>We integrate cross-agent attention blocks into the pretrained video model, which enable interactive transmission of spatial-temporal information across agents.
  arXiv  Detail & Related papers  (2026-03-03T07:41:12Z)
• Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective [54.77404771454794]
  We develop a flexible and robust world model for Multi-Agent Reinforcement Learning (MARL) using diffusion models.<n>Our method, Diffusion-Inspired Multi-Agent world model (DIMA), achieves state-of-the-art performance across multiple multi-agent control benchmarks.
  arXiv  Detail & Related papers  (2025-05-27T09:11:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.