Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach

• URL: http://arxiv.org/abs/2307.01316v2
• Date: Thu, 13 Jul 2023 14:41:32 GMT
• Title: Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach
• Authors: Iman Sharifi, Mustafa Yildirim, Saber Fallah
• Abstract summary: This paper introduces a novel neuro-symbolic model-free DRL approach, called DRL with Symbolic Logics (DRLSL) It combines the strengths of DRL (learning from experience) and symbolic first-order logics (knowledge-driven reasoning) to enable safe learning in real-time interactions of autonomous driving within real environments. We have implemented the DRLSL framework in autonomous driving using the highD dataset and demonstrated that our method successfully avoids unsafe actions during both the training and testing phases.
• Score: 6.961253535504979
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The dynamic nature of driving environments and the presence of diverse road users pose significant challenges for decision-making in autonomous driving. Deep reinforcement learning (DRL) has emerged as a popular approach to tackle this problem. However, the application of existing DRL solutions is mainly confined to simulated environments due to safety concerns, impeding their deployment in real-world. To overcome this limitation, this paper introduces a novel neuro-symbolic model-free DRL approach, called DRL with Symbolic Logics (DRLSL) that combines the strengths of DRL (learning from experience) and symbolic first-order logics (knowledge-driven reasoning) to enable safe learning in real-time interactions of autonomous driving within real environments. This innovative approach provides a means to learn autonomous driving policies by actively engaging with the physical environment while ensuring safety. We have implemented the DRLSL framework in autonomous driving using the highD dataset and demonstrated that our method successfully avoids unsafe actions during both the training and testing phases. Furthermore, our results indicate that DRLSL achieves faster convergence during training and exhibits better generalizability to new driving scenarios compared to traditional DRL methods.

Related papers

• CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving [45.05135725542318]
  We propose a framework that enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. By combining RL and imitation, we demonstrate that our method achieves state-of-the-art results in closed loop simulation driving benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T07:31:29Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving [11.072917563013428]
  Reinforcement learning (RL) has been widely used in decision-making and control tasks, but the risk is very high for the agent in the training process. In this paper, we propose a novel algorithm based on the long and short-term constraints (LSTC) for safe RL. The proposed method achieves higher safety in continuous state and action tasks, and exhibits higher exploration performance in long-distance decision-making tasks.
  arXiv  Detail & Related papers  (2024-03-27T02:41:52Z)
• DRNet: A Decision-Making Method for Autonomous Lane Changingwith Deep Reinforcement Learning [7.2282857478457805]
  "DRNet" is a novel DRL-based framework that enables a DRL agent to learn to drive by executing reasonable lane changing on simulated highways. Our DRL agent has the ability to learn the desired task without causing collisions and outperforms DDQN and other baseline models.
  arXiv  Detail & Related papers  (2023-11-02T21:17:52Z)
• Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving [33.672722472758636]
  offline Reinforcement Learning(RL) approaches exhibit notable efficacy in addressing sequential decision-making problems from offline datasets. We introduce the saFety-aware strUctured Scenario representatION ( Fusion) to facilitate the learning of a generalizable end-to-end driving policy. Empirical evidence in various driving scenarios attests that Fusion significantly enhances the safety and generalizability of autonomous driving agents.
  arXiv  Detail & Related papers  (2023-10-31T18:21:24Z)
• Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration [75.51109230296568]
  We argue that extracting expert policy from offline data to guide online exploration is a promising solution to mitigate the conserveness issue. We propose Guided Online Distillation (GOLD), an offline-to-online safe RL framework. GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms.
  arXiv  Detail & Related papers  (2023-09-18T00:22:59Z)
• FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing [71.76084256567599]
  We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL) Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.
  arXiv  Detail & Related papers  (2023-04-19T17:33:47Z)
• Safe Decision-making for Lane-change of Autonomous Vehicles via Human Demonstration-aided Reinforcement Learning [3.8902094267855167]
  Decision-making is critical for lane change in autonomous driving. Poor runtime safety hinders RL-based decision-making strategies from complex driving tasks in practice. Human demonstrations are incorporated into the RL-based decision-making strategy in this paper.
  arXiv  Detail & Related papers  (2022-07-01T14:16:50Z)
• Learning to be Safe: Deep RL with a Safety Critic [72.00568333130391]
  A natural first approach toward safe RL is to manually specify constraints on the policy's behavior. We propose to learn how to be safe in one set of tasks and environments, and then use that learned intuition to constrain future behaviors.
  arXiv  Detail & Related papers  (2020-10-27T20:53:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.