Momentum: Persistent Dynamics for Video World Models
Abstract Overview
This paper studies how video world models can maintain consistent scene dynamics even when objects leave the current camera view and later reappear. The proposed system, Momentum, represents the world as synchronized videos from graph-connected reference rigs, each containing eight cameras that jointly cover the surroundings. It uses a two-stage pipeline: WorldUpdate generates an evolving multi-view world state across rigs, and WorldInteract conditions target-camera video generation on that state. To support this setting, the authors also build a CARLA-based data-generation and evaluation framework for dynamic grounding in street scenes.
Novelty
The main novelty is treating synchronized multi-view reference videos as an evolving world state rather than relying on static images, observation history, or cached spatial memories. The method also introduces graph-propagated multi-rig generation to coordinate scene dynamics across space and time without explicit object tracking or a static/dynamic scene decomposition.
Results
Using ground-truth reference videos, the dynamically grounded model outperforms static grounding baselines on visual fidelity, camera-trajectory accuracy, and both static and dynamic scene consistency in seen and unseen towns. In the end-to-end setting, Momentum improves dynamic scene consistency (SCS-dyn) by 130.2% in seen towns and 126.2% in an unseen town over the image-conditioned dynamically grounded baseline, while also improving visual fidelity and overall consistency.
Key Points
- Momentum models persistent scene dynamics through synchronized eight-camera reference rigs connected in a graph, enabling world evolution beyond the current target view.
- The framework separates world-state rollout (WorldUpdate) from target-view rendering (WorldInteract), so generated reference videos serve as reusable dynamic grounding for multiple target trajectories.
- Experiments on CARLA street scenes show that dynamic grounding is substantially more effective than static image conditioning for maintaining consistency of moving objects across time and viewpoints.