Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving

• URL: http://arxiv.org/abs/2607.29052v1
• Date: Fri, 31 Jul 2026 06:10:24 GMT
• Title: Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving
• Abstract summary: Vision-Language Models (VLMs) provide explicit reasoning, enhancing both interpretability and driving robustness.<n>We propose a new framework that integrates structured reasoning and geometric precision through a teacher-student architecture.<n>Our proposed solution integrates two goals: providing explicit reasoning for interpretability and delivering robust and accurate driving performance.
• Score: 9.830258914764979
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end (E2E) autonomous driving aims to learn a direct mapping from visual observations to control actions. However, these E2E models often act as black boxes and struggle with complex scenarios. To address this, recent works incorporate Vision-Language Models (VLMs) to provide explicit reasoning, enhancing both interpretability and driving robustness. These approaches typically rely on pre-generated annotations, which suffer from potentially flawed labels and require costly human labor. In this work, we propose a new framework that integrates structured reasoning and geometric precision through a teacher-student architecture. The teacher model introduces reflective reasoning, where the VLM generates logical explanations and then reflectively refines the reasoning under the supervision of ground-truth action. This enhances zero-shot generalization without intermediate labels. The student model distills the teacher's reasoning capabilities via supervised fine-tuning. We also design a separate waypoint decoder that interprets textual reasoning into continuous trajectories. Our proposed solution integrates two goals: providing explicit reasoning for interpretability and delivering robust and accurate driving performance. It leverages the synergy between these two objectives within a staged inference engine to enhance driving performance and explicitly uses the reasoning to guide driving prediction. Evaluated on Waymo benchmarks, our framework outperforms classical reasoning-based baselines in zero-shot reasoning, waypoint accuracy, and inference efficiency. Our experiments validate this design, demonstrating that the reasoning text makes a significant contribution to driving inference, resulting in around a 24% improvement in performance compared to an identical model that lacks reasoning. Our work advances reasoning-driven autonomous driving toward interpretable and deployable systems.

Related papers

• Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving [79.71689985927625]
  We propose IRR-Drive, an adaptive multimodal reflection framework for autonomous driving.<n>IRR-Drive explicitly models anticipated scene evolution, enabling the model to rigorously self-correct and refine its initial intent.<n>Our method achieves state-of-the-art performance on the benchmark in both PDMS and NAVSIMS.
  arXiv  Detail & Related papers  (2026-06-22T06:53:58Z)
• EvoDriveVLA: Evolving Autonomous Driving Vision-Language-Action Model via Collaborative Perception-Planning Distillation [58.84721000276226]
  EvoDriveVLA is a novel collaborative perception-planning framework.<n>It integrates self-anchored perceptual constraints and oracle-guided trajectory optimization.<n>EvoDriveVLA achieves SOTA performance in open-loop evaluation and significantly enhances performance in closed-loop evaluation.
  arXiv  Detail & Related papers  (2026-03-10T10:19:07Z)
• Structured Reasoning for Large Language Models [59.215789462977206]
  We propose Structured Reasoning (SCR), a framework that decouples reasoning trajectories into explicit, evaluable, and trainable components.<n>SCR substantially improves reasoning efficiency and self-verification.<n>Compared with existing reasoning paradigms, it reduces output token length by up to 50%.
  arXiv  Detail & Related papers  (2026-01-12T04:04:01Z)
• Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning [71.19675094463834]
  This work introduces Counterfactual VLA, a self-reflective VLA framework that enables the model to reason about and revise its planned actions before execution.<n>CF-VLA first generates time-segmented meta-actions that summarize driving intent, and then performs counterfactual reasoning conditioned on both the meta-actions and the visual context.<n> Experiments on large-scale driving datasets show that CF-VLA improves trajectory accuracy by up to 17.6%, enhances safety metrics by 20.5%, and exhibits adaptive thinking.
  arXiv  Detail & Related papers  (2025-12-30T19:04:17Z)
• Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future [125.92052530850425]
  Vision-Language-Action (VLA) frameworks integrate perception with language-grounded decision making.<n>VLA frameworks offer a pathway toward more interpretable, generalizable, and human-aligned driving policies.<n>This work aims to establish a coherent foundation for advancing human-compatible autonomous driving systems.
  arXiv  Detail & Related papers  (2025-12-18T16:57:44Z)
• MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning [51.20229133553804]
  Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL)<n>Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning.<n>We propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters.<n>By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions.
  arXiv  Detail & Related papers  (2025-12-15T18:31:32Z)
• Latent Chain-of-Thought World Modeling for End-to-End Driving [45.726304769312414]
  We present Latent-CoT-Drive (LCDrive), a model that expresses CoT in a latent language.<n>Our approach unifies CoT reasoning and decision making by representing both in an action-aligned latent space.<n>On a large-scale end-to-end driving benchmark, LCDrive achieves faster inference, better trajectory quality, and larger improvements from interactive reinforcement learning.
  arXiv  Detail & Related papers  (2025-12-11T02:22:07Z)
• CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving [10.836513600206118]
  We propose Chain-of-Thought (CoT) reasoning for autonomous driving to enhance both numerical and causal reasoning in Vision-Language Models (VLMs)<n>CoT4AD integrates visual observations and language instructions to perform semantic reasoning, scene understanding, and trajectory planning.<n>Experiments on both real-world and simulated benchmarks, including nuScenes and Bench2Drive, demonstrate that CoT4AD achieves state-of-the-art performance in both open-loop and closed-loop evaluations.
  arXiv  Detail & Related papers  (2025-11-27T15:13:13Z)
• DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving [22.293019898794963]
  We present AutoDriveRL, a unified training framework that formulates autonomous driving as a structured reasoning process over four core tasks.<n>Within this framework, we train DriveRX, a cross-task reasoning VLM designed for real-time decision-making.<n>Our analysis highlights the impact of vision encoder design and reward-guided reasoning compression.
  arXiv  Detail & Related papers  (2025-05-27T03:21:04Z)
• ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Driving [26.169506297338234]
  multimodal large language models (MLLMs) have garnered significant attention in the field of end-to-end (E2E) autonomous driving.<n>We propose ReasonPlan, a novel MLLM fine-tuning framework designed for closed-loop driving through holistic reasoning.<n>Our method outperforms the mainstream E2E imitation learning method by a large margin of 19% L2 and 16.1 driving score on Bench2Drive benchmark.
  arXiv  Detail & Related papers  (2025-05-26T14:12:38Z)
• ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models [9.316712964093506]
  Vision-language models (VLMs) show promise for autonomous driving but often lack transparent reasoning capabilities that are critical for safety.<n>We investigate whether explicitly modeling reasoning during fine-tuning enhances VLM performance on driving decision tasks.
  arXiv  Detail & Related papers  (2025-04-14T23:16:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.