Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving

• URL: http://arxiv.org/abs/2603.25740v1
• Date: Thu, 26 Mar 2026 17:59:54 GMT
• Title: Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving
• Abstract summary: Drive My Way (DMW) is a personalized driving framework that aligns with users' long-term driving habits and adapts to real-time user instructions.<n>DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning.<n>Our data and code are available at https://dmw-cvpr.io/.
• Score: 18.914134572789703
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human driving behavior is inherently personal, which is shaped by long-term habits and influenced by short-term intentions. Individuals differ in how they accelerate, brake, merge, yield, and overtake across diverse situations. However, existing end-to-end autonomous driving systems either optimize for generic objectives or rely on fixed driving modes, lacking the ability to adapt to individual preferences or interpret natural language intent. To address this gap, we propose Drive My Way (DMW), a personalized Vision-Language-Action (VLA) driving framework that aligns with users' long-term driving habits and adapts to real-time user instructions. DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning, while natural language instructions provide additional short-term guidance. Closed-loop evaluation on the Bench2Drive benchmark demonstrates that DMW improves style instruction adaptation, and user studies show that its generated behaviors are recognizable as each driver's own style, highlighting personalization as a key capability for human-centered autonomous driving. Our data and code are available at https://dmw-cvpr.github.io/.

Related papers

• STAGE: STyle-controllable Action GEneration for personalized autonomous driving [15.187069954986343]
  Driving style refers to the behavioral preferences that drivers maintain during driving.<n>We develop a style controllable action generation method, STAGE, for driving tasks.<n>We reveal the unique functionalities of STAGE, including its style controllability, style continuity, driving style alignment capability and driving safety.
  arXiv  Detail & Related papers  (2026-07-31T15:17:50Z)
• A Large-Language-Model Supported Personalized Driving Framework for Lane Change in Highway Scenarios [1.6314656571060535]
  This paper proposes a large language model (LLM)-supported personalized driving framework for highway lane-change scenarios.<n>The framework maps natural-language driving commands to executable planning parameters in the open-source Apollo automated driving stack.<n>Experiments show that the derived parameter sets generate distinguishable personalized lane-change behaviors, while RAG consistently improves preference interpretation.
  arXiv  Detail & Related papers  (2026-06-30T10:58:21Z)
• Vega: Learning to Drive with Natural Language Instructions [93.49357278917223]
  We propose a unified Vision-Language-World-Action model, Vega, for instruction-based generation and planning.<n>We employ the autoregressive paradigm to process visual inputs (vision) and language instructions (language) and the diffusion paradigm to generate future predictions.<n>Our method achieves superior planning performance and exhibits strong instruction-following abilities.
  arXiv  Detail & Related papers  (2026-03-26T17:59:56Z)
• Driving with A Thousand Faces: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Driving [39.31712441721641]
  Person2Drive is a comprehensive personalized E2E-AD platform and benchmark.<n>It includes an open-source, flexible data collection system that simulates realistic scenarios to generate personalized driving datasets.<n>Our dataset and code will be released after acceptance.
  arXiv  Detail & Related papers  (2026-02-21T08:42:32Z)
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future [125.92052530850425]
  Vision-Language-Action (VLA) frameworks integrate perception with language-grounded decision making.<n>VLA frameworks offer a pathway toward more interpretable, generalizable, and human-aligned driving policies.<n>This work aims to establish a coherent foundation for advancing human-compatible autonomous driving systems.
  arXiv  Detail & Related papers  (2025-12-18T16:57:44Z)
• MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning [51.20229133553804]
  Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL)<n>Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning.<n>We propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters.<n>By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions.
  arXiv  Detail & Related papers  (2025-12-15T18:31:32Z)
• E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving [56.50212124887739]
  We introduce Open-Domain End-to-End (OD-E2E) autonomous driving, where an autonomous vehicle (AV) must interpret free-form natural-language commands, infer the emotion, and plan a physically feasible trajectory.<n>We propose E3AD, an emotion-aware VLA framework that augments semantic understanding with two cognitively inspired components.<n>A consistency-oriented training scheme, combining modality pretraining with preference-based alignment, further enforces coherence between emotional intent and driving actions.
  arXiv  Detail & Related papers  (2025-12-04T12:17:25Z)
• StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Driving [7.525510086747996]
  Personalization has been largely overlooked in the context of end-to-end autonomous driving (E2EAD)<n>We introduce the first large-scale real-world dataset explicitly curated for personalized E2EAD.<n>We introduce the first standardized benchmark for systematically evaluating personalized E2EAD models.
  arXiv  Detail & Related papers  (2025-06-30T15:48:38Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving [9.637200409973804]
  Existing end-to-end driving approaches often rely on predefined driving styles or require continuous user feedback for adaptation.<n>We propose a novel learning (MORL) approach for autonomous driving that supports dynamic, context-dependent preferences.
  arXiv  Detail & Related papers  (2025-05-08T13:16:37Z)
• On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validation [17.085548386025412]
  Vision-Language Models (VLMs) offer promising solutions to personalized driving. We propose a lightweight yet effective on-board VLM framework that provides low-latency personalized driving performance. Our system has demonstrated the ability to provide safe, comfortable, and personalized driving experiences across various scenarios.
  arXiv  Detail & Related papers  (2024-11-17T23:20:37Z)
• Learning Accurate and Human-Like Driving using Semantic Maps and Attention [152.48143666881418]
  This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. We exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. Our models are trained and evaluated on the Drive360 + HERE dataset, which features 60 hours and 3000 km of real-world driving data.
  arXiv  Detail & Related papers  (2020-07-10T22:25:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.