Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging

• URL: http://arxiv.org/abs/2212.00618v1
• Date: Thu, 1 Dec 2022 16:14:40 GMT
• Title: Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging
• Authors: Soumith Udatha, Yiwei Lyu, John Dolan
• Abstract summary: We use control barrier functions embedded into the reinforcement learning policy to optimize the performance of the autonomous driving vehicle. The proposed algorithm is not just a safe ramp merging algorithm but a safe autonomous driving algorithm applied to address ramp merging on highways.
• Score: 7.103977648997475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prior work has looked at applying reinforcement learning and imitation learning approaches to autonomous driving scenarios, but either the safety or the efficiency of the algorithm is compromised. With the use of control barrier functions embedded into the reinforcement learning policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle. However, control barrier functions need a good approximation of the model of the car. We use probabilistic control barrier functions as an estimate of the model uncertainty. The algorithm is implemented as an online version in the CARLA (Dosovitskiy et al., 2017) Simulator and as an offline version on a dataset extracted from the NGSIM Database. The proposed algorithm is not just a safe ramp merging algorithm but a safe autonomous driving algorithm applied to address ramp merging on highways.

Related papers

• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  Multimodal Large Language Models (MLLMs) can process both visual and textual data. We propose SafeAuto, a novel framework that enhances MLLM-based autonomous driving systems by incorporating both unstructured and structured knowledge.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• Automatic driving lane change safety prediction model based on LSTM [3.8749946206111603]
  The trajectory prediction method based on LSTM network has obvious advantages in predicting the trajectory in the long time domain. The research results show that compared with the traditional model-based method, the trajectory prediction method based on LSTM network has obvious advantages in predicting the trajectory in the long time domain.
  arXiv  Detail & Related papers  (2024-02-28T12:34:04Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Tackling Real-World Autonomous Driving using Deep Reinforcement Learning [63.3756530844707]
  In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts acceleration and steering angle. In order to deploy the system on board the real self-driving car, we also develop a module represented by a tiny neural network.
  arXiv  Detail & Related papers  (2022-07-05T16:33:20Z)
• Learning Optimal Antenna Tilt Control Policies: A Contextual Linear Bandit Approach [65.27783264330711]
  Controlling antenna tilts in cellular networks is imperative to reach an efficient trade-off between network coverage and capacity. We devise algorithms learning optimal tilt control policies from existing data. We show that they can produce optimal tilt update policy using much fewer data samples than naive or existing rule-based learning algorithms.
  arXiv  Detail & Related papers  (2022-01-06T18:24:30Z)
• Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles [13.40143623056186]
  This paper proposes a safe RL algorithm for optimal control of nonlinear systems with time-varying state and control constraints. A multi-step policy evaluation mechanism is proposed to predict the policy's safety risk under time-varying safety constraints and guide the policy to update safely. The proposed algorithm outperforms several state-of-the-art RL algorithms in the simulated Safety Gym environment.
  arXiv  Detail & Related papers  (2021-12-18T10:45:31Z)
• Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations [50.37808220291108]
  This paper addresses learning safe output feedback control laws from partial observations of expert demonstrations. We first propose robust output control barrier functions (ROCBFs) as a means to guarantee safety. We then formulate an optimization problem to learn ROCBFs from expert demonstrations that exhibit safe system behavior.
  arXiv  Detail & Related papers  (2021-11-18T23:21:00Z)
• Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving [1.995792341399967]
  We develop a safe decision-making method for self-driving cars in a multi-lane, single-agent setting. The proposed approach utilizes deep reinforcement learning to achieve a high-level policy for safe tactical decision-making.
  arXiv  Detail & Related papers  (2021-05-13T19:17:30Z)
• A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning [23.007323699176467]
  We propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinator for the low-level controllers. Safety is guaranteed by the low-level optimization/sampling-based controllers, while the high-level reinforcement learning algorithm makes H-CtRL an adaptive and efficient behavior planner. The proposed H-CtRL is proved to be effective in various realistic simulation scenarios, with satisfying performance in terms of both safety and efficiency.
  arXiv  Detail & Related papers  (2021-01-17T20:45:42Z)
• Driving-Policy Adaptive Safeguard for Autonomous Vehicles Using Reinforcement Learning [19.71676985220504]
  This paper proposes a driving-policy adaptive safeguard (DPAS) design, including a collision avoidance strategy and an activation function. The driving-policy adaptive activation function should dynamically assess current driving policy risk and kick in when an urgent threat is detected. The results are calibrated by naturalistic driving data and show that the proposed safeguard reduces the collision rate significantly without introducing more interventions.
  arXiv  Detail & Related papers  (2020-12-02T08:01:53Z)
• Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems [81.7983463275447]
  Learning-based control algorithms require data collection with abundant supervision for training. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
  arXiv  Detail & Related papers  (2020-05-09T05:57:43Z)
• Learning Control Barrier Functions from Expert Demonstrations [69.23675822701357]
  We propose a learning based approach to safe controller synthesis based on control barrier functions (CBFs) We analyze an optimization-based approach to learning a CBF that enjoys provable safety guarantees under suitable Lipschitz assumptions on the underlying dynamical system. To the best of our knowledge, these are the first results that learn provably safe control barrier functions from data.
  arXiv  Detail & Related papers  (2020-04-07T12:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.