OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

• URL: http://arxiv.org/abs/2604.17915v1
• Date: Mon, 20 Apr 2026 07:50:00 GMT
• Title: OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models
• Abstract summary: We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
• Score: 69.2503510410147
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Models(VLMs) excel at autoregressive text generation, yet end-to-end autonomous driving requires multi-task learning with structured outputs and heterogeneous decoding behaviors, such as autoregressive language generation, parallel object detection and trajectory regression. To accommodate these differences, existing systems typically introduce separate or cascaded decoders, resulting in architectural fragmentation and limited backbone reuse. In this work, we present a unified autonomous driving framework built upon a pretrained VLM, where heterogeneous decoding behaviors are reconciled within a single transformer decoder. We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling. By organizing visual and structured query tokens within a single causal decoder, structured queries can naturally condition on visual context through the original attention mechanism. Textual and structured outputs share a common attention backbone, enabling stable joint optimization across heterogeneous tasks. Trajectory planning is realized within the same causal LLM decoder by introducing structured trajectory queries. This unified formulation enables planning to share the pretrained attention backbone with images and perception tokens. Extensive experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance, including 0.28 L2 and 0.18 collision rate on nuScenes open-loop evaluation and competitive results (86.8 PDMS) on NAVSIM closed-loop evaluation. The full model preserves multi-modal generation capability, while an efficient inference mode achieves approximately 40% lower latency. Code and models are available at https://github.com/Z1zyw/OneDrive

Related papers

• StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models [84.17128030384099]
  StreamPI is a streaming multimodal temporal modeling framework.<n>It equips single-frame VLA with temporal reasoning capability without introducing any additional parameters.<n> Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.
  arXiv  Detail & Related papers  (2026-08-26T17:33:19Z)
• DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving [39.60553851234952]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving.<n>We propose DriveVLA-M0, a retrieval-augmented VLA with failure-aware latent memory.<n>We show that DriveVLA-M0 scales effectively with additional memory, enabling training-free performance gains through memory expansion.
  arXiv  Detail & Related papers  (2026-08-11T03:01:46Z)
• UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling [57.126059929740926]
  We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface.<n>We propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient visual auto-regressive decoder.<n>In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families.
  arXiv  Detail & Related papers  (2026-07-27T08:35:09Z)
• OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMs [31.589945976149973]
  We introduce OmniTrace, a lightweight and model-agnostic framework that formalizes attribution as a generation-time tracing problem.<n>We show that generation-aware span-level attribution produces more stable and interpretable explanations than naive self-attribution.<n>Our results suggest that treating attribution as a structured generation-time tracing problem provides a scalable foundation for transparency in omni-modal language models.
  arXiv  Detail & Related papers  (2026-03-20T17:25:00Z)
• Unifying Language-Action Understanding and Generation for Autonomous Driving [25.23561391638388]
  Vision-Language-Action (VLA) models are emerging as a promising paradigm for end-to-end autonomous driving.<n>Existing methods suffer from two critical limitations: a persistent misalignment between language instructions and action outputs, and the inherent inefficiency of typical auto-regressive action generation.<n>We introduce LinkVLA, a novel architecture that directly addresses these challenges to enhance both alignment and efficiency.
  arXiv  Detail & Related papers  (2026-03-02T04:41:10Z)
• Accelerating Structured Chain-of-Thought in Autonomous Vehicles [55.78214218065611]
  Chain-of-Thought (CoT) reasoning enhances the decision-making capabilities of vision-language-action models in autonomous driving.<n>FastDriveCoT is a novel parallel decoding method that accelerates template-structured CoT.
  arXiv  Detail & Related papers  (2026-02-02T22:14:26Z)
• ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving [44.008287454538596]
  Vision-language models (VLMs) enrich this paradigm by introducing cross-modal priors and commonsense reasoning.<n>Current VLM-based planners face three key challenges: (i) a mismatch between discrete text reasoning and continuous control, (ii) high latency from autoregressive chain-of-thought decoding, and (iii) inefficient or non-causal planners that limit real-time deployment.<n>We propose ColaVLA, a unified vision-language-action framework that transfers reasoning from text to a unified latent space and couples it with a hierarchical, parallel trajectory decoder.
  arXiv  Detail & Related papers  (2025-12-28T14:06:37Z)
• FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling [5.609178055761294]
  We present FSDAM, a framework that achieves joint attention prediction and caption generation with 100 annotated examples.<n> FSDAM achieves competitive performance on attention prediction, generates coherent, and context-aware explanations.<n>This work shows that effective attention-conditioned generation is achievable with limited supervision, opening new possibilities for practical deployment of explainable driver attention systems.
  arXiv  Detail & Related papers  (2025-11-16T17:50:30Z)
• OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation [91.45421429922506]
  OneCAT is a unified multimodal model that seamlessly integrates understanding, generation, and editing.<n>Our framework eliminates the need for external components such as Vision Transformers (ViT) or vision tokenizer during inference.
  arXiv  Detail & Related papers  (2025-09-03T17:29:50Z)
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling [84.51372201195132]
  CronusVLA is a unified framework that extends single-frame VLA models to the multi-frame paradigm.<n>CronusVLA achieves leading performance and superior robustness, with a 70.9% success rate.<n>These results highlight the potential of efficient multi-frame adaptation in VLA models for more powerful and robust real-world deployment.
  arXiv  Detail & Related papers  (2025-06-24T17:30:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.