Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving

• URL: http://arxiv.org/abs/2603.24581v1
• Date: Wed, 25 Mar 2026 17:56:07 GMT
• Title: Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
• Abstract summary: Latent-WAM is an efficient end-to-end autonomous driving framework.<n>It achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations.
• Score: 40.17041348571413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations. Existing world-model-based planners suffer from inadequately compressed representations, limited spatial understanding, and underutilized temporal dynamics, resulting in sub-optimal planning under constrained data and compute budgets. Latent-WAM addresses these limitations with two core modules: a Spatial-Aware Compressive World Encoder (SCWE) that distills geometric knowledge from a foundation model and compresses multi-view images into compact scene tokens via learnable queries, and a Dynamic Latent World Model (DLWM) that employs a causal Transformer to autoregressively predict future world status conditioned on historical visual and motion representations. Extensive experiments on NAVSIM v2 and HUGSIM demonstrate new state-of-the-art results: 89.3 EPDMS on NAVSIM v2 and 28.9 HD-Score on HUGSIM, surpassing the best prior perception-free method by 3.2 EPDMS with significantly less training data and a compact 104M-parameter model.

Related papers

• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning [18.658825757342544]
  Vision-language models (VLMs) have shown strong performance on static visual understanding.<n>They still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion.<n>We propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model.
  arXiv  Detail & Related papers  (2026-04-29T17:48:01Z)
• DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning [44.543763428623976]
  We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning.<n>The proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and latency controllable.
  arXiv  Detail & Related papers  (2026-04-02T08:33:18Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• Kinematics-Aware Latent World Models for Data-Efficient Autonomous Driving [8.391716372682316]
  Vehicle kinematic information is incorporated into the observation encoder to ground latent transitions in physically meaningful motion dynamics.<n> geometry-aware supervision regularizes the RSSM latent state to capture task-relevant spatial structure beyond pixel reconstruction.
  arXiv  Detail & Related papers  (2026-03-07T15:47:54Z)
• Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory [101.2076718776139]
  We propose a robust interactive world model capable of maintaining coherent visual memory over 1000+ frames in complex real-world environments.<n>We introduce a Pose-free Memory (HPMC) that distills historical latents into a fixed-budget geometric representation.<n>We also propose an Uncertainty-aware Action Labeling module that discretizes continuous motion into a tri-state logic.
  arXiv  Detail & Related papers  (2026-02-02T17:52:56Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• SAMPO:Scale-wise Autoregression with Motion PrOmpt for generative world models [42.814012901180774]
  textbfSAMPO is a hybrid framework that combines visual autoregressive modeling for intra-frame generation with causal modeling for next-frame generation.<n>We show that SAMPO achieves competitive performance in action-conditioned video prediction and model-based control.<n>We also evaluate SAMPO's zero-shot generalization and scaling behavior, demonstrating its ability to generalize to unseen tasks.
  arXiv  Detail & Related papers  (2025-09-19T02:41:37Z)
• LaDi-WM: A Latent Diffusion-based World Model for Predictive Manipulation [45.02469804709771]
  We propose LaDi-WM, a world model that predicts the latent space of future states using diffusion modeling.<n>We show that LaDi-WM significantly enhances policy performance by 27.9% on the LIBERO-LONG benchmark and 20% on the real-world scenario.
  arXiv  Detail & Related papers  (2025-05-13T04:42:14Z)
• World Model-Based Learning for Long-Term Age of Information Minimization in Vehicular Networks [53.98633183204453]
  In this paper, a novel world model-based learning framework is proposed to minimize packet-completeness-aware age of information (CAoI) in a vehicular network.<n>A world model framework is proposed to jointly learn a dynamic model of the mmWave V2X environment and use it to imagine trajectories for learning how to perform link scheduling.<n>In particular, the long-term policy is learned in differentiable imagined trajectories instead of environment interactions.
  arXiv  Detail & Related papers  (2025-05-03T06:23:18Z)
• Simplifying Latent Dynamics with Softly State-Invariant World Models [10.722955763425228]
  We introduce the Parsimonious Latent Space Model (PLSM), a world model that regularizes the latent dynamics to make the effect of the agent's actions more predictable. We find that our regularization improves accuracy, generalization, and performance in downstream tasks.
  arXiv  Detail & Related papers  (2024-01-31T13:52:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.