Code World Model: Coding Agent as World Brain

• URL: http://arxiv.org/abs/2608.25927v1
• Date: Wed, 26 Aug 2026 15:37:33 GMT
• Title: Code World Model: Coding Agent as World Brain
• Abstract summary: We introduce Code World Model, a framework that separates world evolution from visual realization.<n>A coding agent serves as the world brain, reasoning about events and their consequences.<n>To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wisetemporal constraints into a proxy video.
• Score: 51.31807545747191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.

Related papers

• Programmable World Model [37.43654073749753]
  We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation.<n>An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules.<n>A lightweight engine executes these programs to update and maintain an explicit, persistent global world state.
  arXiv  Detail & Related papers  (2026-09-09T17:59:32Z)
• AlayaWorld: Long-Horizon and Playable Video World Generation [36.52656377442462]
  We present textbfAlayaWorld, a full-stack open-source framework for building interactive generative worlds.<n>AlayaWorld enables open-ended real-time interaction, allowing users to freely navigate and perform diverse actions such as combat, spell casting, and monster summoning.
  arXiv  Detail & Related papers  (2026-07-07T13:59:57Z)
• LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models [32.92934803081681]
  Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control.<n>They implicitly assume that the world only evolves within the observer's field of view.<n>Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime.<n>We propose LiveWorld, a novel framework that extends video world models to support persistent world evolution.
  arXiv  Detail & Related papers  (2026-03-07T10:31:39Z)
• VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation [23.86958300272144]
  We introduce VDAWorld, a framework where a Vision-Language Model (VLM) acts as an intelligent agent to orchestrate this process.<n>The VLM autonomously constructs a grounded (2D or 3D) scene representation by selecting from a suite of vision tools.<n>It can then infer latent dynamics from the static scene to predict plausible future states.
  arXiv  Detail & Related papers  (2025-12-11T19:21:47Z)
• PAN: A World Model for General, Interactable, and Long-Horizon World Simulation [49.805071498152536]
  We introduce PAN, a general, interactable, and long-horizon world model.<n>It predicts future world states through high-quality video simulation conditioned on history and natural language actions.<n>Experiments show that PAN achieves strong performance in action-conditioned world simulation, long-horizon forecasting, and simulative reasoning.
  arXiv  Detail & Related papers  (2025-11-12T07:20:35Z)
• Simulating the Visual World with Artificial Intelligence: A Roadmap [48.64639618440864]
  Video generation is shifting from generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.<n>This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components.<n>We trace the progression of video generation through four generations, culminating in a video generation model that embodies intrinsic physical plausibility.
  arXiv  Detail & Related papers  (2025-11-11T18:59:50Z)
• Can World Models Benefit VLMs for World Dynamics? [59.73433292793044]
  We investigate the capabilities when world model priors are transferred into Vision-Language Models.<n>We name our best-performing variant Dynamic Vision Aligner (DyVA)<n>We find DyVA to surpass both open-source and proprietary baselines, achieving state-of-the-art or comparable performance.
  arXiv  Detail & Related papers  (2025-10-01T13:07:05Z)
• iVideoGPT: Interactive VideoGPTs are Scalable World Models [70.02290687442624]
  World models empower model-based agents to interactively explore, reason, and plan within imagined environments for real-world decision-making. This work introduces Interactive VideoGPT, a scalable autoregressive transformer framework that integrates multimodal signals--visual observations, actions, and rewards--into a sequence of tokens. iVideoGPT features a novel compressive tokenization technique that efficiently discretizes high-dimensional visual observations.
  arXiv  Detail & Related papers  (2024-05-24T05:29:12Z)
• WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens [75.02160668328425]
  We introduce WorldDreamer, a pioneering world model to foster a comprehensive comprehension of general world physics and motions. WorldDreamer frames world modeling as an unsupervised visual sequence modeling challenge. Our experiments show that WorldDreamer excels in generating videos across different scenarios, including natural scenes and driving environments.
  arXiv  Detail & Related papers  (2024-01-18T14:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.