ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving

• URL: http://arxiv.org/abs/2505.24317v1
• Date: Fri, 30 May 2025 08:00:51 GMT
• Title: ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving
• Authors: Yongming Chen, Miner Chen, Liewen Liao, Mingyang Jiang, Xiang Zuo, Hengrui Zhang, Yuchen Xi, Songan Zhang,
• Abstract summary: This study introduces a responsibility-oriented reward function that explicitly incorporates traffic regulations into theReinforcement learning framework.<n>We introduce a Traffic Regulation Knowledge Graph and leveraged Vision-Language Models alongside Retrieval-Augmented Generation techniques to automate reward assignment.
• Score: 6.713954449470747
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Reinforcement learning (RL) in autonomous driving employs a trial-and-error mechanism, enhancing robustness in unpredictable environments. However, crafting effective reward functions remains challenging, as conventional approaches rely heavily on manual design and demonstrate limited efficacy in complex scenarios. To address this issue, this study introduces a responsibility-oriented reward function that explicitly incorporates traffic regulations into the RL framework. Specifically, we introduced a Traffic Regulation Knowledge Graph and leveraged Vision-Language Models alongside Retrieval-Augmented Generation techniques to automate reward assignment. This integration guides agents to adhere strictly to traffic laws, thus minimizing rule violations and optimizing decision-making performance in diverse driving conditions. Experimental validations demonstrate that the proposed methodology significantly improves the accuracy of assigning accident responsibilities and effectively reduces the agent's liability in traffic incidents.

Related papers

• Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RL [11.198097218885191]
  This work introduces a novel student-teacher framework for automatic curriculum learning.<n>The teacher, a graph-based multi-agent RL component, adaptively generates traffic behaviors across diverse difficulty levels.<n>Results demonstrate the teacher's ability to generate diverse traffic behaviors.
  arXiv  Detail & Related papers  (2025-07-25T10:35:30Z)
• Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning [11.602831593017427]
  This paper addresses the challenges of training end-to-end autonomous driving agents using Reinforcement Learning (RL)<n>RL agents are typically trained in a fixed set of scenarios and nominal behavior of surrounding road users in simulations.<n>We propose an automatic curriculum learning framework that dynamically generates driving scenarios with adaptive complexity based on the agent's evolving capabilities.
  arXiv  Detail & Related papers  (2025-05-13T06:26:57Z)
• Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving [10.950036191948605]
  Reinforcement Learning (RL) is a promising approach for achieving autonomous driving due to robust decision-making capabilities.<n>We introduce a novel risk-aware objective for various driving interactions based on a two-dimensional ellipsoid function.<n>We evaluate the efficacy of our proposed reward in unsignalized intersection scenarios with varying traffic densities.
  arXiv  Detail & Related papers  (2025-05-10T19:05:03Z)
• Towards a Reward-Free Reinforcement Learning Framework for Vehicle Control [1.5883812630616523]
  Reinforcement learning plays a crucial role in vehicle control by guiding agents to learn optimal control strategies.<n>In vehicle control applications, rewards typically need to be manually designed while considering multiple implicit factors.<n>We propose a reward-free reinforcement learning framework (RFRLF) to address these issues.
  arXiv  Detail & Related papers  (2025-02-21T07:43:24Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching [81.82487256783674]
  GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching.<n>This paper introduces GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching.
  arXiv  Detail & Related papers  (2024-08-19T08:23:38Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Reinforcement Learning with Model Predictive Control for Highway Ramp Metering [14.389086937116582]
  This work explores the synergy between model-based and learning-based strategies to enhance traffic flow management.<n>The control problem is formulated as an RL task by crafting a suitable stage cost function.<n>An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp.
  arXiv  Detail & Related papers  (2023-11-15T09:50:54Z)
• A Multiplicative Value Function for Safe and Efficient Reinforcement Learning [131.96501469927733]
  We propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. We evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations.
  arXiv  Detail & Related papers  (2023-03-07T18:29:15Z)
• Unified Automatic Control of Vehicular Systems with Reinforcement Learning [64.63619662693068]
  This article contributes a streamlined methodology for vehicular microsimulation. It discovers high performance control strategies with minimal manual design. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering.
  arXiv  Detail & Related papers  (2022-07-30T16:23:45Z)
• Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
  This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing. We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
  arXiv  Detail & Related papers  (2022-06-28T17:08:31Z)
• UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning [1.1339580074756188]
  Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. This motivates the use of model-based offline RL approaches, which leverage planning.
  arXiv  Detail & Related papers  (2021-11-22T10:37:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.