Learning Vision-Language-Action World Models for Autonomous Driving

• URL: http://arxiv.org/abs/2604.09059v1
• Date: Fri, 10 Apr 2026 07:38:05 GMT
• Title: Learning Vision-Language-Action World Models for Autonomous Driving
• Abstract summary: Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving.<n>VLA models often lack explicit modeling of temporal dynamics and global world consistency.<n>We present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning.
• Score: 15.103497388527943
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and control within a unified multimodal framework. However, they often lack explicit modeling of temporal dynamics and global world consistency, which limits their foresight and safety. In contrast, world models can simulate plausible future scenes but generally struggle to reason about or evaluate the imagined future they generate. In this work, we present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning to improve driving foresight. VLA-World first uses an action-derived feasible trajectory to guide the generation of the next-frame image, capturing rich spatial and temporal cues that describe how the surrounding environment evolves. The model then reasons over this self-generated future imagined frame to refine the predicted trajectory, achieving higher performance and better interpretability. To support this pipeline, we curate nuScenes-GR-20K, a generative reasoning dataset derived from nuScenes, and employ a three-stage training strategy that includes pretraining, supervised fine-tuning, and reinforcement learning. Extensive experiments demonstrate that VLA-World consistently surpasses state-of-the-art VLA and world-model baselines on both planning and future-generation benchmarks. Project page: https://vlaworld.github.io

Related papers

• HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving [11.703169226106288]
  We propose HyWorldVLA, a hybrid world-VLA framework that unifies pixel-level supervision and latent representation learning.<n>In the pre-training stage, HyWorldVLA predicts latent video encoded by a pre-trained video VAE, while simultaneously reconstructing video frames to provide precise pixel-level grounding.<n>During the subsequent co-fine-tuning phase, the model exclusively predicts latent features, which are fed into an action expert to generate trajectories.
  arXiv  Detail & Related papers  (2026-07-23T07:11:41Z)
• LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model [16.87741001074065]
  Vision-Language-Action (VLA) models have emerged as a promising framework for end-to-end autonomous driving.<n>Recent attempts to incorporate dense visual supervision via world modeling often overemphasize pixel-level image reconstruction.<n>We propose LVDrive, a Latent Visual representation enhanced VLA framework for autonomous driving.
  arXiv  Detail & Related papers  (2026-05-21T07:31:49Z)
• LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving [60.31765454895336]
  We present LMGenDrive, the first framework that combines multimodal understanding with generative world models for end-to-end closed-loop driving.<n>We propose a progressive three-stage training strategy, from vision pretraining to multi-step long-horizon driving, to improve stability and performance.
  arXiv  Detail & Related papers  (2026-04-09T19:13:14Z)
• Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving [52.04950569530877]
  We present Uni-World VLA, a unified vision-language-action model that tightly interleaves future frame prediction and trajectory planning.<n>Our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions.
  arXiv  Detail & Related papers  (2026-03-28T14:39:51Z)
• Chain of World: World Model Thinking in Latent Motion [24.24061036481793]
  Vision-Language-Action (VLA) models often overlook the predictive and temporal-causal structure underlying visual dynamics.<n>We introduce CoWVLA, a new "Chain of World" paradigm that unifies world-model temporal reasoning with a disentangled latent motion representation.<n>CoWVLA outperforms existing world-model and latent-action approaches and achieves moderate computational efficiency.
  arXiv  Detail & Related papers  (2026-03-03T17:52:06Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving [25.94896726674773]
  DriveWorld-VLA is a novel framework that unifies world modeling and planning within a latent space.<n>It supports controllable, action-conditioned imagination at the feature level.<n>It achieves state-of-the-art performance with 91.3 PDMS on NAVSIMv1, 86.8S on NAVSIMv2, and 0.16 3-second average collision rate on nuScenes.
  arXiv  Detail & Related papers  (2026-02-06T09:25:48Z)
• World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy [55.03832008486675]
  World-VLA-Loop is a closed-loop framework for the joint refinement of world models and Vision-Language-Action (VLA) policies.<n>We propose a state-aware video world model that functions as a high-fidelity interactive simulator by jointly predicting future observations and reward signals.
  arXiv  Detail & Related papers  (2026-02-06T08:57:55Z)
• DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving [52.63591791507895]
  We propose textbfDriveVLA-W0, a training paradigm that employs world modeling to predict future images.<n>This task generates a dense, self-supervised signal that compels the model to learn the underlying dynamics of the driving environment.<n>Experiments on the NAVSIM v1/v2 benchmark and a 680x larger in-house dataset demonstrate that DriveVLA-W0 significantly outperforms BEV and VLA baselines.
  arXiv  Detail & Related papers  (2025-10-14T17:59:47Z)
• Can World Models Benefit VLMs for World Dynamics? [59.73433292793044]
  We investigate the capabilities when world model priors are transferred into Vision-Language Models.<n>We name our best-performing variant Dynamic Vision Aligner (DyVA)<n>We find DyVA to surpass both open-source and proprietary baselines, achieving state-of-the-art or comparable performance.
  arXiv  Detail & Related papers  (2025-10-01T13:07:05Z)
• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving [19.81442567260658]
  We propose a visual-temporalT framework that enables VLAs to think in images.<n>On nuScenes and NAVSIM, FSDrive improves accuracy and reduces collisions.
  arXiv  Detail & Related papers  (2025-05-23T09:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.