SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guarantees

• URL: http://arxiv.org/abs/2407.16857v1
• Date: Tue, 23 Jul 2024 21:54:39 GMT
• Title: SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guarantees
• Authors: Tianyu Shi, Ilia Smirnov, Omar ElSamadisy, Baher Abdulhai,
• Abstract summary: SECRM-2D is an RL autonomous driving controller that balances optimization of efficiency and comfort and follows a fixed route. We evaluate SECRM-2D against several learning and non-learning baselines in simulated test scenarios.
• Score: 5.156059061769101
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the last decade, there has been increasing interest in autonomous driving systems. Reinforcement Learning (RL) shows great promise for training autonomous driving controllers, being able to directly optimize a combination of criteria such as efficiency comfort, and stability. However, RL- based controllers typically offer no safety guarantees, making their readiness for real deployment questionable. In this paper, we propose SECRM-2D (the Safe, Efficient and Comfortable RL- based driving Model with Lane-Changing), an RL autonomous driving controller (both longitudinal and lateral) that balances optimization of efficiency and comfort and follows a fixed route, while being subject to hard analytic safety constraints. The aforementioned safety constraints are derived from the criterion that the follower vehicle must have sufficient headway to be able to avoid a crash if the leader vehicle brakes suddenly. We evaluate SECRM-2D against several learning and non-learning baselines in simulated test scenarios, including freeway driving, exiting, merging, and emergency braking. Our results confirm that representative previously-published RL AV controllers may crash in both training and testing, even if they are optimizing a safety objective. By contrast, our controller SECRM-2D is successful in avoiding crashes during both training and testing, improves over the baselines in measures of efficiency and comfort, and is more faithful in following the prescribed route. In addition, we achieve a good theoretical understanding of the longitudinal steady-state of a collection of SECRM-2D vehicles.

Related papers

• Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving [11.072917563013428]
  Reinforcement learning (RL) has been widely used in decision-making and control tasks, but the risk is very high for the agent in the training process. In this paper, we propose a novel algorithm based on the long and short-term constraints (LSTC) for safe RL. The proposed method achieves higher safety in continuous state and action tasks, and exhibits higher exploration performance in long-distance decision-making tasks.
  arXiv  Detail & Related papers  (2024-03-27T02:41:52Z)
• Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems [15.863561935347692]
  We develop provably safe and convergent reinforcement learning algorithms for control of nonlinear dynamical systems. Recent advances at the intersection of control and RL follow a two-stage, safety filter approach to enforcing hard safety constraints. We develop a single-stage, sampling-based approach to hard constraint satisfaction that learns RL controllers enjoying classical convergence guarantees.
  arXiv  Detail & Related papers  (2024-03-06T19:39:20Z)
• CAT: Closed-loop Adversarial Training for Safe End-to-End Driving [54.60865656161679]
  Adversarial Training (CAT) is a framework for safe end-to-end driving in autonomous vehicles. Cat aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios. Cat can effectively generate adversarial scenarios countering the agent being trained.
  arXiv  Detail & Related papers  (2023-10-19T02:49:31Z)
• Evaluation of Safety Constraints in Autonomous Navigation with Deep Reinforcement Learning [62.997667081978825]
  We compare two learnable navigation policies: safe and unsafe. The safe policy takes the constraints into the account, while the other does not. We show that the safe policy is able to generate trajectories with more clearance (distance to the obstacles) and makes less collisions while training without sacrificing the overall performance.
  arXiv  Detail & Related papers  (2023-07-27T01:04:57Z)
• A Multiplicative Value Function for Safe and Efficient Reinforcement Learning [131.96501469927733]
  We propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. We evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations.
  arXiv  Detail & Related papers  (2023-03-07T18:29:15Z)
• Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System [1.8899300124593645]
  Reinforcement learning (RL)-based driver assistance systems seek to improve fuel consumption via continual improvement of powertrain control actions. In this paper, an exponential control barrier function (ECBF) is derived and utilized to filter unsafe actions proposed by an RL-based driver assistance system. The proposed safe-RL scheme is trained and evaluated in car following scenarios where it is shown that it effectively avoids collision both during training and evaluation.
  arXiv  Detail & Related papers  (2023-01-03T00:25:00Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving [100.57791628642624]
  We introduce a safety guaranteed learning framework for vision-based end-to-end autonomous driving. We design a learning system equipped with differentiable control barrier functions (dCBFs) that is trained end-to-end by gradient descent.
  arXiv  Detail & Related papers  (2022-03-04T16:14:33Z)
• Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model [0.0]
  Car following is a prime function in autonomous driving. Recent literature shows that bilateral car following that considers the vehicle ahead and the vehicle behind exhibits better system stability. We propose and introduce a Deep Reinforcement Learning framework for car following control by integrating bilateral information into both state and reward function.
  arXiv  Detail & Related papers  (2022-03-03T17:23:36Z)
• Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic Junction Driving [20.85562165500152]
  In a road traffic junction scenario, the vehicle typically receives partial observations from the transportation environment. In this study, we evaluated the safety performance of three baseline DRL models (DQN, A2C, and PPO) Our proposed self-awareness attention-DQN can significantly improve the safety performance in intersection and roundabout scenarios.
  arXiv  Detail & Related papers  (2022-01-20T11:21:33Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.