DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models

• URL: http://arxiv.org/abs/2510.13108v1
• Date: Wed, 15 Oct 2025 03:00:38 GMT
• Title: DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models
• Authors: Jingyu Song, Zhenxin Li, Shiyi Lan, Xinglong Sun, Nadine Chang, Maying Shen, Joshua Chen, Katherine A. Skinner, Jose M. Alvarez,
• Abstract summary: We introduce DriveCritic, a novel framework featuring two key contributions.<n>The dataset is a curated collection of challenging scenarios where context is critical for correct judgment.<n>The DriveCritic model learns to adjudicate between trajectory pairs by integrating visual and symbolic context.
• Score: 24.168614747778538
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Benchmarking autonomous driving planners to align with human judgment remains a critical challenge, as state-of-the-art metrics like the Extended Predictive Driver Model Score (EPDMS) lack context awareness in nuanced scenarios. To address this, we introduce DriveCritic, a novel framework featuring two key contributions: the DriveCritic dataset, a curated collection of challenging scenarios where context is critical for correct judgment and annotated with pairwise human preferences, and the DriveCritic model, a Vision-Language Model (VLM) based evaluator. Fine-tuned using a two-stage supervised and reinforcement learning pipeline, the DriveCritic model learns to adjudicate between trajectory pairs by integrating visual and symbolic context. Experiments show DriveCritic significantly outperforms existing metrics and baselines in matching human preferences and demonstrates strong context awareness. Overall, our work provides a more reliable, human-aligned foundation to evaluating autonomous driving systems.

Related papers

• From Human Intention to Action Prediction: A Comprehensive Benchmark for Intention-driven End-to-End Autonomous Driving [67.23302649816466]
  Current autonomous driving systems operate at a level of intelligence akin to following simple steering commands.<n>We introduce Intention-Drive, the first comprehensive benchmark designed to evaluate the ability to translate high-level human intent into safe and precise driving actions.
  arXiv  Detail & Related papers  (2025-12-13T11:59:51Z)
• MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving [13.786046699744476]
  We propose MindDrive, a framework that integrates high-quality trajectory generation with comprehensive decision reasoning.<n>In particular, the proposed Future-aware Trajectory Generator (FaTG) performs ego-conditioned "what-if" simulations to predict potential future scenes.<n>Building upon this, the VLM-oriented Evaluator (VLoE) leverages the reasoning capability of a large vision-language model to conduct multi-objective evaluations.
  arXiv  Detail & Related papers  (2025-12-04T04:16:10Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• AutoDrive-R$^2$: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving [37.260140808367716]
  We propose AutoDrive-R$2$, a novel VLA framework that enhances both reasoning and self-reflection capabilities of autonomous driving systems.<n>We first propose an innovative CoT dataset named nuScenesR$2$-6K for supervised fine-tuning.<n>We then employ the Group Relative Policy Optimization (GRPO) algorithm within a physics-grounded reward framework to ensure reliable smoothness and realistic trajectory planning.
  arXiv  Detail & Related papers  (2025-09-02T04:32:24Z)
• ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving [64.12414815634847]
  Vision-Language Models (VLMs) and Driving World Models (DWMs) have independently emerged as powerful recipes addressing different aspects of this challenge.<n>We propose ImagiDrive, a novel end-to-end autonomous driving framework that integrates a VLM-based driving agent with a DWM-based scene imaginer.
  arXiv  Detail & Related papers  (2025-08-15T12:06:55Z)
• StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Driving [7.525510086747996]
  Personalization has been largely overlooked in the context of end-to-end autonomous driving (E2EAD)<n>We introduce the first large-scale real-world dataset explicitly curated for personalized E2EAD.<n>We introduce the first standardized benchmark for systematically evaluating personalized E2EAD models.
  arXiv  Detail & Related papers  (2025-06-30T15:48:38Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning [68.45848423501927]
  We propose a holistic vision-language dataset that aligns agent models with 3D driving tasks through counterfactual reasoning.<n>Our approach enhances decision-making by evaluating potential scenarios and their outcomes, similar to human drivers considering alternative actions.
  arXiv  Detail & Related papers  (2025-04-06T03:54:21Z)
• Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases [102.05741859030951]
  We propose CODA-LM, the first benchmark for the automatic evaluation of LVLMs for self-driving corner cases.<n>We show that using the text-only large language models as judges reveals even better alignment with human preferences than the LVLM judges.<n>Our CODA-VLM performs comparably with GPT-4V, even surpassing GPT-4V by +21.42% on the regional perception task.
  arXiv  Detail & Related papers  (2024-04-16T14:20:55Z)
• Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving [38.28159034562901]
  Reason2Drive is a benchmark dataset with over 600K video-text pairs. We characterize the autonomous driving process as a sequential combination of perception, prediction, and reasoning steps. We introduce a novel aggregated evaluation metric to assess chain-based reasoning performance in autonomous systems.
  arXiv  Detail & Related papers  (2023-12-06T18:32:33Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.