DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving

• URL: http://arxiv.org/abs/2608.10413v1
• Date: Tue, 11 Aug 2026 03:01:46 GMT
• Title: DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving
• Abstract summary: Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving.<n>We propose DriveVLA-M0, a retrieval-augmented VLA with failure-aware latent memory.<n>We show that DriveVLA-M0 scales effectively with additional memory, enabling training-free performance gains through memory expansion.
• Score: 39.60553851234952
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for end-to-end autonomous driving by enabling unified reasoning across perception, language, and planning. However, existing approaches lack mechanisms to exploit past failures or adapt to distribution shifts, causing the model to persistently underperform on similar scenarios where it has previously failed. In this paper, we propose DriveVLA-M0, a retrieval-augmented VLA with failure-aware latent memory. We construct a latent memory pool that stores failure cases along with their structure scene representations and expert trajectory labels, and design a dedicated Retrieve Model that decouples static road structure and dynamic agent interactions to enable structurally grounded retrieval. At inference time, retrieved cases are injected into the model via a lightweight decoupled LoRA-based test-time training (TTT) mechanism, allowing targeted and scenario-specific correction without modifying the backbone. Extensive experiments on NAVSIMv1 and NAVSIMv2 benchmark demonstrate that our approach consistently outperforms prior methods, achieving 94.1 PDMS on Navtest and 47.0 EPDMS on Navhard with only 26.44 ms TTT backward latency overhead. Furthermore, we show that DriveVLA-M0 scales effectively with additional memory, enabling training-free performance gains through memory expansion. The code is available at https://github.com/ZebinX/DriveVLA-M0.

Related papers

• CoLT-Drive: Counterfactual Long-Tail Benchmarking and Knowledge-Preserving Adaptation for Driving Affordance Prediction [27.20821192335003]
  Long-tail autonomous driving failures are often framed as rare-object recognition errors.<n>We formalize this problem as decision-level driving affordance prediction, where a model maps a front-view image, ego-motion history, and navigation command to a structured longitudinal-lateral meta-action.<n>To improve deployable small VLMs, we propose KPA, a knowledge-preserving adaptation framework that combines structured perception-to-decision prompting, SLERP-based expert merging, and RegMoE, a regime-aware LoRA mixture-of-experts module.
  arXiv  Detail & Related papers  (2026-08-31T18:44:30Z)
• Grounded Semantic Re-Binding for Robust Instruction Generalization in Vision-Language-Action Models [20.8047896631941]
  Vision-Language-Action (VLA) models excel in robotic manipulation but suffer catastrophic performance drops when canonical instructions are paraphrased.<n>We show that current VLAs successfully retain the correct task identity internally.<n>We propose Grounded Semantic Re-binding (GSR), an elegant intervention that bypasses unstable joint routing.
  arXiv  Detail & Related papers  (2026-08-03T17:02:04Z)
• DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving [8.267228986831894]
  DriveVer is a lightweight, plug-and-play Test-Time Verifier to enable autonomous driving systems to validate and refine trajectories without costly and heavy training.<n>We show that DriveVer significantly improves the performance of base planning models. Notably, as an extremely compact model with only 34M parameters, DriveVer introduces minimal computational overhead, achieving competitive results.
  arXiv  Detail & Related papers  (2026-07-01T03:50:45Z)
• Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recovery [62.75419724651416]
  We introduce textbfSentinel-VLA, a metacognitive VLA model equipped with an active sentinel'' module to monitor real-time execution status.<n>All training data is automatically generated and annotated through our designed pipeline.<n>Real-world experiments demonstrate that Sentinel-VLA boosts the task success rate by over 30% compared to the SOTA model, PI0.
  arXiv  Detail & Related papers  (2026-05-02T02:10:54Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training [48.70704477452434]
  We introduce textbfCapTrack, a capability-centric framework for analyzing forgetting in large language models.<n>We conduct a large-scale empirical study across post-training algorithms, domains, and model families.<n>We find that forgetting extends beyond parametric knowledge, with pronounced drift in robustness and default behaviors.
  arXiv  Detail & Related papers  (2026-02-19T09:46:24Z)
• TS-Memory: Plug-and-Play Memory for Time Series Foundation Models [63.21390142212087]
  Time Series Foundation Models (TSFMs) achieve strong zero-shot forecasting through large-scale pre-training.<n>Existing solutions face a trade-off: Parametric Adaptation can cause catastrophic forgetting, while Non-Parametric Retrieval improves forecasts but incurs high latency due to datastore search.<n>We propose Parametric Memory Distillation and implement it as TS-Memory, a lightweight memory adapter that augments frozen TSFMs.
  arXiv  Detail & Related papers  (2026-02-12T04:16:19Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• CoReVLA: A Dual-Stage End-to-End Autonomous Driving Framework for Long-Tail Scenarios via Collect-and-Refine [73.74077186298523]
  CoReVLA is a continual learning framework for autonomous driving.<n>It improves the performance in long-tail scenarios through a dual-stage process of data Collection and behavior Refinement.<n>CoReVLA achieves a Driving Score (DS) of 72.18 and a Success Rate (SR) of 50%, outperforming state-of-the-art methods by 7.96 DS and 15% SR under long-tail, safety-critical scenarios.
  arXiv  Detail & Related papers  (2025-09-19T13:25:56Z)
• Deep Bilinear Koopman Model for Real-Time Vehicle Control in Frenet Frame [0.0]
  This paper presents a deep Koopman approach for modeling and control of vehicle dynamics within the curvilinear Frenet frame.<n>The proposed framework uses a deep neural network architecture to simultaneously learn the Koopman operator and its associated invariant subspace from the data.<n>The proposed controller achieved significant reductions in tracking error relative to baseline controllers, confirming its suitability for real-time implementation in embedded autonomous vehicle systems.
  arXiv  Detail & Related papers  (2025-07-16T18:49:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.