Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for
Discretionary Lane Change
- URL: http://arxiv.org/abs/2403.00446v1
- Date: Fri, 1 Mar 2024 11:03:17 GMT
- Title: Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for
Discretionary Lane Change
- Authors: Ruichen Xu, Xiao Liu, Jinming Xu, Yuan Lin
- Abstract summary: How to perform safe and appropriate lane change is a popular topic of research in the field of autonomous driving.
We introduce safe hybrid reinforcementaction learning into discretionary lane change for the first time.
At a traffic density of 15 vehicles per kilometer (15 veh/km), the PASAC-PIDLag algorithm exhibits superior safety with a collision rate of 0%, outperforming the PASAC algorithm, which has a collision rate of 1%.
- Score: 6.221047868628538
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous lane-change, a key feature of advanced driver-assistance systems,
can enhance traffic efficiency and reduce the incidence of accidents. However,
safe driving of autonomous vehicles remains challenging in complex
environments. How to perform safe and appropriate lane change is a popular
topic of research in the field of autonomous driving. Currently, few papers
consider the safety of reinforcement learning in autonomous lane-change
scenarios. We introduce safe hybrid-action reinforcement learning into
discretionary lane change for the first time and propose Parameterized Soft
Actor-Critic with PID Lagrangian (PASAC-PIDLag) algorithm. Furthermore, we
conduct a comparative analysis of the Parameterized Soft Actor-Critic (PASAC),
which is an unsafe version of PASAC-PIDLag. Both algorithms are employed to
train the lane-change strategy of autonomous vehicles to output discrete
lane-change decision and longitudinal vehicle acceleration. Our simulation
results indicate that at a traffic density of 15 vehicles per kilometer (15
veh/km), the PASAC-PIDLag algorithm exhibits superior safety with a collision
rate of 0%, outperforming the PASAC algorithm, which has a collision rate of
1%. The outcomes of the generalization assessments reveal that at low traffic
density levels, both the PASAC-PIDLag and PASAC algorithms are proficient in
attaining a 0% collision rate. Under conditions of high traffic flow density,
the PASAC-PIDLag algorithm surpasses PASAC in terms of both safety and
optimality.
Related papers
- Automatic driving lane change safety prediction model based on LSTM [3.8749946206111603]
The trajectory prediction method based on LSTM network has obvious advantages in predicting the trajectory in the long time domain.
The research results show that compared with the traditional model-based method, the trajectory prediction method based on LSTM network has obvious advantages in predicting the trajectory in the long time domain.
arXiv Detail & Related papers (2024-02-28T12:34:04Z) - CAT: Closed-loop Adversarial Training for Safe End-to-End Driving [54.60865656161679]
Adversarial Training (CAT) is a framework for safe end-to-end driving in autonomous vehicles.
Cat aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios.
Cat can effectively generate adversarial scenarios countering the agent being trained.
arXiv Detail & Related papers (2023-10-19T02:49:31Z) - A Counterfactual Safety Margin Perspective on the Scoring of Autonomous
Vehicles' Riskiness [52.27309191283943]
This paper presents a data-driven framework for assessing the risk of different AVs' behaviors.
We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision.
arXiv Detail & Related papers (2023-08-02T09:48:08Z) - DenseLight: Efficient Control for Large-scale Traffic Signals with Dense
Feedback [109.84667902348498]
Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network.
Most prior TSC methods leverage deep reinforcement learning to search for a control policy.
We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
arXiv Detail & Related papers (2023-06-13T05:58:57Z) - iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed
Multi-Agent Reinforcement Learning [57.24340061741223]
We introduce a distributed multi-agent reinforcement learning (MARL) algorithm that can predict trajectories and intents in dense and heterogeneous traffic scenarios.
Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations.
arXiv Detail & Related papers (2023-06-09T20:12:02Z) - Studying the Impact of Semi-Cooperative Drivers on Overall Highway Flow [76.38515853201116]
Semi-cooperative behaviors are intrinsic properties of human drivers and should be considered for autonomous driving.
New autonomous planners can consider the social value orientation (SVO) of human drivers to generate socially-compliant trajectories.
We present study of implicit semi-cooperative driving where agents deploy a game-theoretic version of iterative best response.
arXiv Detail & Related papers (2023-04-23T16:01:36Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Safe Reinforcement Learning with Probabilistic Control Barrier Functions
for Ramp Merging [7.103977648997475]
We use control barrier functions embedded into the reinforcement learning policy to optimize the performance of the autonomous driving vehicle.
The proposed algorithm is not just a safe ramp merging algorithm but a safe autonomous driving algorithm applied to address ramp merging on highways.
arXiv Detail & Related papers (2022-12-01T16:14:40Z) - Decision-Making under On-Ramp merge Scenarios by Distributional Soft
Actor-Critic Algorithm [10.258474373022075]
We propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called the Shielded Distributional Soft Actor-critic (SDSAC)
The results show that the SDSAC has the best safety performance compared to baseline algorithms and efficient driving simultaneously.
arXiv Detail & Related papers (2021-03-08T03:57:32Z) - SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for
Autonomous Vehicles Using Deep Reinforcement Learning [17.412117389855226]
SAINT-ACC: Setyaf-Aware Intelligent ACC system (SAINT-ACC) is designed to achieve simultaneous optimization of traffic efficiency, driving safety, and driving comfort.
A novel dual RL agent-based approach is developed to seek and adapt the optimal balance between traffic efficiency and driving safety/comfort.
arXiv Detail & Related papers (2021-03-06T14:01:29Z) - Driving-Policy Adaptive Safeguard for Autonomous Vehicles Using
Reinforcement Learning [19.71676985220504]
This paper proposes a driving-policy adaptive safeguard (DPAS) design, including a collision avoidance strategy and an activation function.
The driving-policy adaptive activation function should dynamically assess current driving policy risk and kick in when an urgent threat is detected.
The results are calibrated by naturalistic driving data and show that the proposed safeguard reduces the collision rate significantly without introducing more interventions.
arXiv Detail & Related papers (2020-12-02T08:01:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.