SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving

• URL: http://arxiv.org/abs/2609.03602v1
• Date: Thu, 03 Sep 2026 09:47:51 GMT
• Title: SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
• Abstract summary: We propose SV-WAM, a surround-view world-action model (WAM) that preserves full six-camera observations while maintaining efficient inference.<n>At the core of this design is an action-centered causal mask that prevents action tokens from attending to future-video tokens during joint action-video denoising.<n>Experiments on the closed-loop NAVSIMv2 benchmark and the open-loop nuScenes benchmark demonstrate that SV-WAM achieves state-of-the-art planning performance.
• Score: 6.9856663722260235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World models (WMs) have demonstrated strong potential for end-to-end autonomous driving by learning predictive representations of future scene dynamics. However, generating future videos during inference introduces substantial computational overhead, leading many recent driving WMs to adopt a single front camera as input for efficient deployment. This design restricts spatial coverage in safety-critical maneuvers such as lane changes, merges, and turns. To address this limitation, we propose SV-WAM, a surround-view world-action model (WAM) that preserves full six-camera observations while maintaining efficient inference. SV-WAM leverages future-video prediction as dense training supervision for action learning within a shared generative model, rather than as an inference-time output. At the core of this design is an action-centered causal mask that prevents action tokens from attending to future-video tokens during joint action-video denoising. Consequently, the video branch can be discarded at deployment, enabling efficient action-only planning. Furthermore, we introduce a differentiable drivable-area compliance regularizer that penalizes vehicle-footprint corners approaching or crossing drivable boundaries, improving planning safety and boundary awareness. Extensive experiments on the closed-loop NAVSIMv2 benchmark and the open-loop nuScenes benchmark demonstrate that SV-WAM achieves state-of-the-art planning performance with low inference latency and competitive zero-shot transfer capability.

Related papers

• SimWAM: A Simple World Action Model for End-to-End Autonomous Driving [66.93548079427477]
  We present SimWAM, a simple yet effective WAM that leverages future-video prediction as a training-time supervision signal.<n>An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference.<n>Our SimWAM achieves 91.5 PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes.
  arXiv  Detail & Related papers  (2026-08-07T17:59:09Z)
• UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation [23.675212404889805]
  World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving.<n>Existing cascaded or dual-DiT designs separate video imagination from action prediction.<n>We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Transformer.
  arXiv  Detail & Related papers  (2026-07-06T14:18:18Z)
• Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation [80.18527639791074]
  World action models have shown great promise for autonomous driving and urban navigation.<n>Existing approaches suffer key limitations.<n>We propose Metis, an end-to-end WAM framework that decouples video generation and action prediction.
  arXiv  Detail & Related papers  (2026-06-14T15:40:49Z)
• DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving [19.231942908215174]
  We present DriveWAM, a driving world-action model that adapts a pretrained video diffusion transformer into an autoregressive video-action policy.<n>DriveWAM organizes video and action streams into a unified temporal token sequence and trains them under a joint flow-matching objective.<n>Experiments on NAVSIM and the PhysicalAI-Autonomous-Vehicles benchmark show that DriveWAM achieves strong planning performance.
  arXiv  Detail & Related papers  (2026-05-27T14:36:26Z)
• LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model [16.87741001074065]
  Vision-Language-Action (VLA) models have emerged as a promising framework for end-to-end autonomous driving.<n>Recent attempts to incorporate dense visual supervision via world modeling often overemphasize pixel-level image reconstruction.<n>We propose LVDrive, a Latent Visual representation enhanced VLA framework for autonomous driving.
  arXiv  Detail & Related papers  (2026-05-21T07:31:49Z)
• MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving [62.43744546817599]
  Vision-language--action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings.<n>We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model.<n> MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
  arXiv  Detail & Related papers  (2026-05-13T23:35:14Z)
• DriveVA: Video Action Models are Zero-Shot Drivers [23.675212404889805]
  Generalization is a central challenge in autonomous driving, as real-world scenarios require robust performance under unseen conditions.<n>Recent world-model-based planning methods have shown strong capabilities in scene understanding and multi-modal future prediction.<n>We propose DriveVA, a novel autonomous driving world model that jointly decodes future visual forecasts and action sequences in a shared latent generative process.
  arXiv  Detail & Related papers  (2026-04-05T17:43:16Z)
• Chain of World: World Model Thinking in Latent Motion [24.24061036481793]
  Vision-Language-Action (VLA) models often overlook the predictive and temporal-causal structure underlying visual dynamics.<n>We introduce CoWVLA, a new "Chain of World" paradigm that unifies world-model temporal reasoning with a disentangled latent motion representation.<n>CoWVLA outperforms existing world-model and latent-action approaches and achieves moderate computational efficiency.
  arXiv  Detail & Related papers  (2026-03-03T17:52:06Z)
• ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving [64.12414815634847]
  Vision-Language Models (VLMs) and Driving World Models (DWMs) have independently emerged as powerful recipes addressing different aspects of this challenge.<n>We propose ImagiDrive, a novel end-to-end autonomous driving framework that integrates a VLM-based driving agent with a DWM-based scene imaginer.
  arXiv  Detail & Related papers  (2025-08-15T12:06:55Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• Enhancing End-to-End Autonomous Driving with Latent World Model [78.22157677787239]
  We propose a novel self-supervised learning approach using the LAtent World model (LAW) for end-to-end driving.<n> LAW predicts future scene features based on current features and ego trajectories.<n>This self-supervised task can be seamlessly integrated into perception-free and perception-based frameworks.
  arXiv  Detail & Related papers  (2024-06-12T17:59:21Z)
• Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving [56.381918362410175]
  Drive-WM is the first driving world model compatible with existing end-to-end planning models. Our model generates high-fidelity multiview videos in driving scenes.
  arXiv  Detail & Related papers  (2023-11-29T18:59:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.