Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving

• URL: http://arxiv.org/abs/2306.03220v2
• Date: Fri, 25 Aug 2023 06:50:38 GMT
• Title: Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving
• Authors: Lin-Chi Wu, Zengjie Zhang, Sofie Haesaert, Zhiqiang Ma, and Zhiyong Sun
• Abstract summary: This paper investigates how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. We propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors.
• Score: 6.613838702441967
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, the reward function for an RL agent, which is significant to its performance, is challenging to be determined. The conventional work mainly focuses on rewarding safe driving states but does not incorporate the awareness of risky driving behaviors of the vehicles. In this paper, we investigate how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. Based on the essential requirements that prescribe the safety specifications for general autonomous driving in practice, we propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors. A simulation study in OpenAI Gym indicates the advantage of risk-aware reward shaping for various RL agents. Also, we point out that proximal policy optimization (PPO) is likely to be the best RL method that works with risk-aware reward shaping.

Related papers

• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• DRNet: A Decision-Making Method for Autonomous Lane Changingwith Deep Reinforcement Learning [7.2282857478457805]
  "DRNet" is a novel DRL-based framework that enables a DRL agent to learn to drive by executing reasonable lane changing on simulated highways. Our DRL agent has the ability to learn the desired task without causing collisions and outperforms DDQN and other baseline models.
  arXiv  Detail & Related papers  (2023-11-02T21:17:52Z)
• Robust Driving Policy Learning with Guided Meta Reinforcement Learning [49.860391298275616]
  We introduce an efficient method to train diverse driving policies for social vehicles as a single meta-policy. By randomizing the interaction-based reward functions of social vehicles, we can generate diverse objectives and efficiently train the meta-policy. We propose a training strategy to enhance the robustness of the ego vehicle's driving policy using the environment where social vehicles are controlled by the learned meta-policy.
  arXiv  Detail & Related papers  (2023-07-19T17:42:36Z)
• FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing [71.76084256567599]
  We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL) Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.
  arXiv  Detail & Related papers  (2023-04-19T17:33:47Z)
• Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System [1.8899300124593645]
  Reinforcement learning (RL)-based driver assistance systems seek to improve fuel consumption via continual improvement of powertrain control actions. In this paper, an exponential control barrier function (ECBF) is derived and utilized to filter unsafe actions proposed by an RL-based driver assistance system. The proposed safe-RL scheme is trained and evaluated in car following scenarios where it is shown that it effectively avoids collision both during training and evaluation.
  arXiv  Detail & Related papers  (2023-01-03T00:25:00Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Learning to Drive Using Sparse Imitation Reinforcement Learning [0.5076419064097732]
  We propose a hybrid end-to-end control policy that combines the sparse expert driving knowledge with reinforcement learning (RL) policy. We experimentally validate the efficacy of proposed SIRL approach in a complex urban scenario within the CARLA simulator.
  arXiv  Detail & Related papers  (2022-05-24T15:03:11Z)
• How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies [1.496194593196997]
  This paper proposes SafeDQN, which allows to make the behavior of autonomous vehicles safe and interpretable while still being efficient. We show that SafeDQN finds interpretable and safe driving policies for a variety of scenarios and demonstrate how state-of-the-art saliency techniques can help to assess both risk and utility.
  arXiv  Detail & Related papers  (2022-03-16T05:51:22Z)
• Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving [20.85562165500152]
  In a road traffic junction scenario, the vehicle typically receives partial observations from the transportation environment. In this study, we evaluated the safety performance of three baseline DRL models (DQN, A2C, and PPO) Our proposed self-awareness attention-DQN can significantly improve the safety performance in intersection and roundabout scenarios.
  arXiv  Detail & Related papers  (2022-01-20T11:21:33Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)
• Safe Reinforcement Learning via Curriculum Induction [94.67835258431202]
  In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. Existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations. This paper presents an alternative approach inspired by human teaching, where an agent learns under the supervision of an automatic instructor.
  arXiv  Detail & Related papers  (2020-06-22T10:48:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.