Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving

• URL: http://arxiv.org/abs/2505.06737v1
• Date: Sat, 10 May 2025 19:05:03 GMT
• Title: Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving
• Authors: Ahmed Abouelazm, Jonas Michel, Helen Gremmelmaier, Tim Joseph, Philip Schörner, J. Marius Zöllner,
• Abstract summary: Reinforcement Learning (RL) is a promising approach for achieving autonomous driving due to robust decision-making capabilities.<n>We introduce a novel risk-aware objective for various driving interactions based on a two-dimensional ellipsoid function.<n>We evaluate the efficacy of our proposed reward in unsignalized intersection scenarios with varying traffic densities.
• Score: 10.950036191948605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement Learning (RL) is a promising approach for achieving autonomous driving due to robust decision-making capabilities. RL learns a driving policy through trial and error in traffic scenarios, guided by a reward function that combines the driving objectives. The design of such reward function has received insufficient attention, yielding ill-defined rewards with various pitfalls. Safety, in particular, has long been regarded only as a penalty for collisions. This leaves the risks associated with actions leading up to a collision unaddressed, limiting the applicability of RL in real-world scenarios. To address these shortcomings, our work focuses on enhancing the reward formulation by defining a set of driving objectives and structuring them hierarchically. Furthermore, we discuss the formulation of these objectives in a normalized manner to transparently determine their contribution to the overall reward. Additionally, we introduce a novel risk-aware objective for various driving interactions based on a two-dimensional ellipsoid function and an extension of Responsibility-Sensitive Safety (RSS) concepts. We evaluate the efficacy of our proposed reward in unsignalized intersection scenarios with varying traffic densities. The approach decreases collision rates by 21\% on average compared to baseline rewards and consistently surpasses them in route progress and cumulative reward, demonstrating its capability to promote safer driving behaviors while maintaining high-performance levels.

Related papers

• ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving [6.713954449470747]
  This study introduces a responsibility-oriented reward function that explicitly incorporates traffic regulations into theReinforcement learning framework.<n>We introduce a Traffic Regulation Knowledge Graph and leveraged Vision-Language Models alongside Retrieval-Augmented Generation techniques to automate reward assignment.
  arXiv  Detail & Related papers  (2025-05-30T08:00:51Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Uniformly Safe RL with Objective Suppression for Multi-Constraint Safety-Critical Applications [73.58451824894568]
  The widely adopted CMDP model constrains the risks in expectation, which makes room for dangerous behaviors in long-tail states. In safety-critical domains, such behaviors could lead to disastrous outcomes. We propose Objective Suppression, a novel method that adaptively suppresses the task reward maximizing objectives according to a safety critic.
  arXiv  Detail & Related papers  (2024-02-23T23:22:06Z)
• A Counterfactual Safety Margin Perspective on the Scoring of Autonomous Vehicles' Riskiness [52.27309191283943]
  This paper presents a data-driven framework for assessing the risk of different AVs' behaviors. We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision.
  arXiv  Detail & Related papers  (2023-08-02T09:48:08Z)
• Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving [6.613838702441967]
  This paper investigates how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. We propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors.
  arXiv  Detail & Related papers  (2023-06-05T20:10:36Z)
• A Multiplicative Value Function for Safe and Efficient Reinforcement Learning [131.96501469927733]
  We propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. We evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations.
  arXiv  Detail & Related papers  (2023-03-07T18:29:15Z)
• Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving [20.85562165500152]
  In a road traffic junction scenario, the vehicle typically receives partial observations from the transportation environment. In this study, we evaluated the safety performance of three baseline DRL models (DQN, A2C, and PPO) Our proposed self-awareness attention-DQN can significantly improve the safety performance in intersection and roundabout scenarios.
  arXiv  Detail & Related papers  (2022-01-20T11:21:33Z)
• Risk-Constrained Interactive Safety under Behavior Uncertainty for Autonomous Driving [0.0]
  Safety envelopes which restrict the allowed planning region yield interpretable safety under the presence of behavior uncertainty. Humans balance safety and efficiency in dense traffic by accepting a probabilistic risk of violating the safety envelope.
  arXiv  Detail & Related papers  (2021-02-05T08:33:39Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)
• Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learning [16.69903761648675]
  We propose a generic risk-aware DQN approach to learn high level actions for driving through unsignalized intersections. The proposed state representation provides lane based information which allows to be used for multi-lane scenarios. We also propose a risk based reward function which punishes risky situations instead of only collision failures.
  arXiv  Detail & Related papers  (2020-04-09T09:44:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.