KnowVal: A Knowledge-Augmented and Value-Guided Autonomous Driving System

• URL: http://arxiv.org/abs/2512.20299v1
• Date: Tue, 23 Dec 2025 12:08:00 GMT
• Title: KnowVal: A Knowledge-Augmented and Value-Guided Autonomous Driving System
• Authors: Zhongyu Xia, Wenhao Chen, Yongtao Wang, Ming-Hsuan Yang,
• Abstract summary: We propose KnowVal, a new autonomous driving system that enables visual-language reasoning.<n>We construct a comprehensive driving knowledge graph that encodes traffic laws, defensive driving principles, and ethical norms.<n>KnowVal achieves the lowest collision rate on nuScenes and state-of-the-art results on Bench2Drive.
• Score: 44.93698347738791
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Visual-language reasoning, driving knowledge, and value alignment are essential for advanced autonomous driving systems. However, existing approaches largely rely on data-driven learning, making it difficult to capture the complex logic underlying decision-making through imitation or limited reinforcement rewards. To address this, we propose KnowVal, a new autonomous driving system that enables visual-language reasoning through the synergistic integration of open-world perception and knowledge retrieval. Specifically, we construct a comprehensive driving knowledge graph that encodes traffic laws, defensive driving principles, and ethical norms, complemented by an efficient LLM-based retrieval mechanism tailored for driving scenarios. Furthermore, we develop a human-preference dataset and train a Value Model to guide interpretable, value-aligned trajectory assessment. Experimental results show that our method substantially improves planning performance while remaining compatible with existing architectures. Notably, KnowVal achieves the lowest collision rate on nuScenes and state-of-the-art results on Bench2Drive.

Related papers

• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• VLAD: A VLM-Augmented Autonomous Driving Framework with Hierarchical Planning and Interpretable Decision Process [40.3578745624081]
  We propose a vision-language autonomous driving model, which integrates a fine-tuned Visual Language Models (VLMs) with a state-of-the-art end-to-end system.<n>We implement a specialized fine-tuning approach using custom question-answer datasets designed specifically to improve the spatial reasoning capabilities of the model.<n>Our system produces interpretable natural language explanations of driving decisions, thereby increasing transparency and trustworthiness of the traditionally black-box end-to-end architecture.
  arXiv  Detail & Related papers  (2025-07-02T01:52:40Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving [51.47621083057114]
  SOLVE is an innovative framework that synergizes Vision-Language Models with end-to-end (E2E) models to enhance autonomous vehicle planning.<n>Our approach emphasizes knowledge sharing at the feature level through a shared visual encoder, enabling comprehensive interaction between VLM and E2E components.
  arXiv  Detail & Related papers  (2025-05-22T15:44:30Z)
• RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Driving [10.984203470464687]
  Vision-language models (VLMs) often suffer from limitations such as inadequate spatial perception and hallucination.<n>We propose a retrieval-augmented decision-making (RAD) framework to enhance VLMs' capabilities to reliably generate meta-actions in autonomous driving scenes.<n>We fine-tune VLMs on a dataset derived from the NuScenes dataset to enhance their spatial perception and bird's-eye view image comprehension capabilities.
  arXiv  Detail & Related papers  (2025-03-18T03:25:57Z)
• Learning to Drive by Imitating Surrounding Vehicles [0.8902959815221526]
  We study a data augmentation strategy that leverages the observed trajectories of nearby vehicles as additional demonstrations.<n>We introduce a simple vehicle-selection sampling and filtering strategy that prioritizes informative and diverse driving behaviors.<n>Specifically, the approach reduces collision rates and improves safety metrics compared to the baseline.
  arXiv  Detail & Related papers  (2025-03-08T00:40:47Z)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  We propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge.<n>To explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic.<n>Our Multimodal Retrieval-Augmented Generation model leverages video, control signals, and environmental attributes to learn from past driving experiences.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• LeapVAD: A Leap in Autonomous Driving via Cognitive Perception and Dual-Process Thinking [13.898774643126174]
  LeapVAD implements a human-attentional mechanism to identify and focus on critical traffic elements that influence driving decisions.<n>System consists of an Analytic Process (System-II) that accumulates driving experience through logical reasoning and a Heuristic Process (System-I) that refines this knowledge via fine-tuning and few-shot learning.
  arXiv  Detail & Related papers  (2025-01-14T14:49:45Z)
• Tackling Real-World Autonomous Driving using Deep Reinforcement Learning [63.3756530844707]
  In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts acceleration and steering angle. In order to deploy the system on board the real self-driving car, we also develop a module represented by a tiny neural network.
  arXiv  Detail & Related papers  (2022-07-05T16:33:20Z)
• Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning [13.699336307578488]
  Deep imitative reinforcement learning approach (DIRL) achieves agile autonomous racing using visual inputs. We validate our algorithm both in a high-fidelity driving simulation and on a real-world 1/20-scale RC-car with limited onboard computation.
  arXiv  Detail & Related papers  (2021-07-18T00:00:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.