Safety-aware Causal Representation for Trustworthy Offline Reinforcement
Learning in Autonomous Driving
- URL: http://arxiv.org/abs/2311.10747v3
- Date: Tue, 12 Mar 2024 21:01:38 GMT
- Title: Safety-aware Causal Representation for Trustworthy Offline Reinforcement
Learning in Autonomous Driving
- Authors: Haohong Lin, Wenhao Ding, Zuxin Liu, Yaru Niu, Jiacheng Zhu, Yuming
Niu, Ding Zhao
- Abstract summary: offline Reinforcement Learning(RL) approaches exhibit notable efficacy in addressing sequential decision-making problems from offline datasets.
We introduce the saFety-aware strUctured Scenario representatION ( Fusion) to facilitate the learning of a generalizable end-to-end driving policy.
Empirical evidence in various driving scenarios attests that Fusion significantly enhances the safety and generalizability of autonomous driving agents.
- Score: 33.672722472758636
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In the domain of autonomous driving, the offline Reinforcement Learning~(RL)
approaches exhibit notable efficacy in addressing sequential decision-making
problems from offline datasets. However, maintaining safety in diverse
safety-critical scenarios remains a significant challenge due to long-tailed
and unforeseen scenarios absent from offline datasets. In this paper, we
introduce the saFety-aware strUctured Scenario representatION (FUSION), a
pioneering representation learning method in offline RL to facilitate the
learning of a generalizable end-to-end driving policy by leveraging structured
scenario information. FUSION capitalizes on the causal relationships between
the decomposed reward, cost, state, and action space, constructing a framework
for structured sequential reasoning in dynamic traffic environments. We conduct
extensive evaluations in two typical real-world settings of the distribution
shift in autonomous vehicles, demonstrating the good balance between safety
cost and utility reward compared to the current state-of-the-art safe RL and IL
baselines. Empirical evidence in various driving scenarios attests that FUSION
significantly enhances the safety and generalizability of autonomous driving
agents, even in the face of challenging and unseen environments. Furthermore,
our ablation studies reveal noticeable improvements in the integration of
causal representation into the offline safe RL algorithm. Our code
implementation is available at: https://sites.google.com/view/safe-fusion/.
Related papers
- SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems.
LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning.
We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
arXiv Detail & Related papers (2023-11-28T03:13:09Z) - DRNet: A Decision-Making Method for Autonomous Lane Changingwith Deep
Reinforcement Learning [7.2282857478457805]
"DRNet" is a novel DRL-based framework that enables a DRL agent to learn to drive by executing reasonable lane changing on simulated highways.
Our DRL agent has the ability to learn the desired task without causing collisions and outperforms DDQN and other baseline models.
arXiv Detail & Related papers (2023-11-02T21:17:52Z) - Guided Online Distillation: Promoting Safe Reinforcement Learning by
Offline Demonstration [75.51109230296568]
We argue that extracting expert policy from offline data to guide online exploration is a promising solution to mitigate the conserveness issue.
We propose Guided Online Distillation (GOLD), an offline-to-online safe RL framework.
GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms.
arXiv Detail & Related papers (2023-09-18T00:22:59Z) - Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep
Reinforcement Learning Approach [6.961253535504979]
This paper introduces a novel neuro-symbolic model-free DRL approach, called DRL with Symbolic Logics (DRLSL)
It combines the strengths of DRL (learning from experience) and symbolic first-order logics (knowledge-driven reasoning) to enable safe learning in real-time interactions of autonomous driving within real environments.
We have implemented the DRLSL framework in autonomous driving using the highD dataset and demonstrated that our method successfully avoids unsafe actions during both the training and testing phases.
arXiv Detail & Related papers (2023-07-03T19:43:21Z) - Online Safety Property Collection and Refinement for Safe Deep
Reinforcement Learning in Mapless Navigation [79.89605349842569]
We introduce the Collection and Refinement of Online Properties (CROP) framework to design properties at training time.
CROP employs a cost signal to identify unsafe interactions and use them to shape safety properties.
We evaluate our approach in several robotic mapless navigation tasks and demonstrate that the violation metric computed with CROP allows higher returns and lower violations over previous Safe DRL approaches.
arXiv Detail & Related papers (2023-02-13T21:19:36Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Differentiable Control Barrier Functions for Vision-based End-to-End
Autonomous Driving [100.57791628642624]
We introduce a safety guaranteed learning framework for vision-based end-to-end autonomous driving.
We design a learning system equipped with differentiable control barrier functions (dCBFs) that is trained end-to-end by gradient descent.
arXiv Detail & Related papers (2022-03-04T16:14:33Z) - Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic
Junction Driving [20.85562165500152]
In a road traffic junction scenario, the vehicle typically receives partial observations from the transportation environment.
In this study, we evaluated the safety performance of three baseline DRL models (DQN, A2C, and PPO)
Our proposed self-awareness attention-DQN can significantly improve the safety performance in intersection and roundabout scenarios.
arXiv Detail & Related papers (2022-01-20T11:21:33Z) - UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning
Leveraging Planning [1.1339580074756188]
Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data.
Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set.
This motivates the use of model-based offline RL approaches, which leverage planning.
arXiv Detail & Related papers (2021-11-22T10:37:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.