Programmable World Model

• URL: http://arxiv.org/abs/2609.10540v1
• Date: Wed, 09 Sep 2026 17:59:32 GMT
• Title: Programmable World Model
• Abstract summary: We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation.<n>An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules.<n>A lightweight engine executes these programs to update and maintain an explicit, persistent global world state.
• Score: 37.43654073749753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

Related papers

• Code World Model: Coding Agent as World Brain [51.31807545747191]
  We introduce Code World Model, a framework that separates world evolution from visual realization.<n>A coding agent serves as the world brain, reasoning about events and their consequences.<n>To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wisetemporal constraints into a proxy video.
  arXiv  Detail & Related papers  (2026-08-26T15:37:33Z)
• PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives [86.3073067035531]
  Video world models simulate future states conditioned on current observations and user actions.<n>Recent systems have demonstrated impressive video consistency and action controllability over long sequences.<n>But fairly comparing these interactive models remains challenging.<n>We employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives.
  arXiv  Detail & Related papers  (2026-08-13T17:59:30Z)
• Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers [50.3000377325823]
  We present WorldWeaver, a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers.<n>We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text.<n>Experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.
  arXiv  Detail & Related papers  (2026-07-23T17:59:58Z)
• LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models [32.92934803081681]
  Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control.<n>They implicitly assume that the world only evolves within the observer's field of view.<n>Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime.<n>We propose LiveWorld, a novel framework that extends video world models to support persistent world evolution.
  arXiv  Detail & Related papers  (2026-03-07T10:31:39Z)
• WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World [100.68103378427567]
  Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fail physically or behaviorally.<n>We introduce WorldLens, a full-spectrum benchmark evaluating how well a model builds, understands, and behaves within its generated world.<n>We further construct WorldLens-26K, a large-scale dataset of human-annotated videos with numerical scores and textual rationales, and develop WorldLens-Agent.
  arXiv  Detail & Related papers  (2025-12-11T18:59:58Z)
• Simulating the Visual World with Artificial Intelligence: A Roadmap [48.64639618440864]
  Video generation is shifting from generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.<n>This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components.<n>We trace the progression of video generation through four generations, culminating in a video generation model that embodies intrinsic physical plausibility.
  arXiv  Detail & Related papers  (2025-11-11T18:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.