Daily Paper Briefing

Abstract Overview

Programmable World Model is a framework for interactive video generation that separates world-state evolution from visual observation synthesis. A coding agent converts natural-language specifications into executable programs that define entity states, relations, and transition rules, while a lightweight engine maintains a persistent canonical world state, including off-screen and non-visual information. The maintained state is represented with state-augmented 3D oriented bounding boxes and deterministically compiled into pixel-aligned identity, semantic, and motion-direction controls for a pretrained video generator. The paper also introduces an automatic training-data pipeline for recovering structured supervision from videos and a benchmark, CombatStateBench, to evaluate whether generated videos remain consistent with engine-maintained states over interactions.

Novelty

The paper's main distinction is its explicit separation of programmable state execution from generative rendering, rather than relying on a video model to implicitly remember and update world facts. It also proposes state-augmented 3D oriented bounding boxes plus a deterministic state compiler as an intermediate interface between executable world programs and video generation, and introduces CombatStateBench for evaluating this setting.

Results

On CombatStateBench, the method reports 94% Count Accuracy and 98% State Accuracy, substantially exceeding the compared interactive video world models in world-state consistency. It also achieves the best reported VBench-based video quality metrics among the compared methods and is shown qualitatively to support long-horizon generation, off-screen entity persistence, and multiple scene types such as combat and racing.

Key Points

1. The system uses an agent and lightweight execution engine to maintain an explicit, persistent world state governed by executable rules derived from natural-language instructions.
2. State-augmented 3D oriented bounding boxes are compiled into pixel-aligned identity, semantic, and direction controls that condition a pretrained video renderer.
3. Experiments on the newly introduced CombatStateBench indicate markedly improved agreement between rendered videos and engine-maintained states, especially for visible character counts and death-state realization.

References

• arXiv: https://arxiv.org/abs/2609.10540v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2609.10540v1
• Hugging Face Papers: https://huggingface.co/papers/2609.10540
• GitHub: https://github.com/AlayaLab/pwm
• Project: https://alaya-lab.github.io/pwm