Long-WAM: Scaling the Context of World-Action Models
Abstract Overview
Long-WAM is a model-and-systems framework for scaling the visual context of causal world-action models under real-time robot control constraints. The paper argues that simply accessing longer history is insufficient; longer context is much more useful when the video foundation model is pretrained autoregressively so that history-to-future structure is already learned. The method first learns causal prediction from large-scale robot and egocentric video without action labels, then adapts this foundation to action generation while preserving causal dependencies and conditioning actions on predicted future latents. The work also pairs the model design with deployment optimizations such as streaming observation encoding, asynchronous execution, and hardware-specific acceleration to keep inference fast enough for online control.
Novelty
The distinctive contribution is the joint claim that context scaling in world-action models depends on causal predictive pretraining, not just larger memory windows. The paper also combines this modeling idea with a real-time deployment stack that preserves future prediction rather than removing it for speed.
Results
Empirically, increasing context improves performance substantially for autoregressively pretrained variants, including a rise on RoboCasa GR-1 from 63.3% to 78.7% as context grows from 0.0 to 19.2 seconds, while a bidirectionally pretrained initialization shows no comparable net gain. Long-WAM reports the best compared results on LIBERO-Long, RoboTwin 2.0, and DOMINO, and demonstrates strong real-world performance, including 95% success on dynamic cup stacking and 81.7% average success on long-horizon YAM tasks. The optimized system runs in real time on several NVIDIA platforms, with 107.4 ms per action chunk on RTX 5090 including future-video latent prediction.
Key Points
- Autoregressive video pretraining is presented as the key factor that lets longer observation history translate into better control, whereas bidirectional pretraining does not show the same long-context benefit.
- The method uses a predict-then-act pipeline in which actions are conditioned on both observed history and predicted future latents, preserving causal visual structure during world-action adaptation.
- A substantial systems contribution supports deployment: asynchronous execution, streaming VAE encoding, quantization, KV reuse, and device-specific tuning reduce latency while retaining predictive conditioning.