SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

• URL: http://arxiv.org/abs/2608.07468v2
• Date: Tue, 11 Aug 2026 08:27:03 GMT
• Title: SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
• Abstract summary: We present SimWAM, a simple yet effective WAM that leverages future-video prediction as a training-time supervision signal.<n>An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference.<n>Our SimWAM achieves 91.5 PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes.
• Score: 66.93548079427477
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World-Action Models (WAMs) improve end-to-end autonomous driving by transferring video dynamics priors to action prediction, but existing methods incur costly test-time future imagination. We present SimWAM, a simple yet effective WAM that leverages future-video prediction as a training-time supervision signal. It co-trains a pretrained video expert and a lightweight action expert with joint flow matching. An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference. Since the two experts share no parameters and interact only through a unified attention interface, the video backbone could be replaced and the action expert scaled independently without modifying the learning objective or inference pipeline. We further apply reinforcement learning to optimize a compositional driving reward beyond trajectory imitation. Our SimWAM achieves 91.5 PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes. These results position SimWAM as a simple yet solid baseline that could readily benefit from advances in video generation for efficient autonomous driving. The code and model weights are available at https://github.com/H-EmbodVis/SimWAM/.

Related papers

• DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning [28.673173377342593]
  General robot manipulation requires policies that can anticipate how visual scenes evolve while executing language instructions.<n>DeVA is a Decoupled Video-Action model with specialized video and action experts, multi-level feature transfer, and physically guidance.<n> Experiments on both simulation benchmarks and real-world deployment demonstrate strong performance with limited data, faster convergence than a unified architecture, and clear performance gains from physical guidance.
  arXiv  Detail & Related papers  (2026-07-27T08:37:35Z)
• UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation [23.675212404889805]
  World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving.<n>Existing cascaded or dual-DiT designs separate video imagination from action prediction.<n>We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Transformer.
  arXiv  Detail & Related papers  (2026-07-06T14:18:18Z)
• Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation [80.18527639791074]
  World action models have shown great promise for autonomous driving and urban navigation.<n>Existing approaches suffer key limitations.<n>We propose Metis, an end-to-end WAM framework that decouples video generation and action prediction.
  arXiv  Detail & Related papers  (2026-06-14T15:40:49Z)
• CausalDrive: Real-time Causal World Models for Autonomous Driving [60.66609721457312]
  We present CausalDrive, a controllable, real-time foundation driving world.<n>CaulDrive operates solely on the initial front-view frame, the ego-vehicle's trajectory, and a macroscopic text prompt.<n>We demonstrate its versatility across three downstream applications: (1) generative closed-loop evaluation with significantly mitigated collision artifacts, (2) large-scale Reinforcement Learning (RL) post-training driven by a Video2Reward module, and (3) real-time human-in-the-loop simulation.
  arXiv  Detail & Related papers  (2026-06-13T15:04:44Z)
• LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving [60.31765454895336]
  We present LMGenDrive, the first framework that combines multimodal understanding with generative world models for end-to-end closed-loop driving.<n>We propose a progressive three-stage training strategy, from vision pretraining to multi-step long-horizon driving, to improve stability and performance.
  arXiv  Detail & Related papers  (2026-04-09T19:13:14Z)
• TrackMAE: Video Representation Learning via Track Mask and Predict [53.79942817343784]
  Masked video modeling (MVM) has emerged as a simple and scalable self-supervised pretraining paradigm.<n>We propose TrackMAE, a simple masked video modeling paradigm that explicitly uses motion information as a reconstruction signal.<n>We evaluate on six datasets across diverse downstream settings and find that TrackMAE consistently outperforms state-of-the-art video self-supervised learning baselines.
  arXiv  Detail & Related papers  (2026-03-28T13:35:23Z)
• S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresight [42.690473567612116]
  Video action models (VAMs) have emerged as a promising paradigm for robot learning.<n>We propose S-VAM, a shortcut video-action model that foresees coherent geometric and semantic representations via a single forward pass.
  arXiv  Detail & Related papers  (2026-03-17T07:21:28Z)
• Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos [20.73513310337503]
  Ego-centric driving videos available online provide an abundant source of visual data for autonomous driving.<n>We propose a label-free, teacher-guided framework for learning autonomous driving representations directly from unposed videos.
  arXiv  Detail & Related papers  (2026-02-25T16:38:53Z)
• MAD: Motion Appearance Decoupling for efficient Driving World Models [94.40548866741791]
  We propose an efficient adaptation framework that converts generalist video models into controllable driving world models.<n>Key idea is to decouple motion learning from appearance synthesis.<n>Scaling to LTX, our MAD-LTX model outperforms all open-source competitors.
  arXiv  Detail & Related papers  (2026-01-14T12:52:23Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• VaViM and VaVAM: Autonomous Driving through Video Generative Modeling [88.33638585518226]
  We introduce an open-source auto-regressive video model (VaM) and its companion video-action model (VaVAM) to investigate how video pre-training transfers to real-world driving.<n>We evaluate our models in open- and closed-loop driving scenarios, revealing that video-based pre-training holds promise for autonomous driving.
  arXiv  Detail & Related papers  (2025-02-21T18:56:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.