Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea

• URL: http://arxiv.org/abs/2402.08502v2
• Date: Thu, 16 May 2024 21:14:14 GMT
• Title: Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea
• Authors: Hanna Krasowski, Matthias Althoff,
• Abstract summary: Reinforcement learning (RL) is a promising method to find motion plans for autonomous vehicles. Our approach accomplishes guaranteed rule-compliance by integrating temporal logic specifications into RL. In numerical evaluations on critical maritime traffic situations, our agent always complies with the formalized legal rules and never collides.
• Score: 8.017543518311196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For safe operation, autonomous vehicles have to obey traffic rules that are set forth in legal documents formulated in natural language. Temporal logic is a suitable concept to formalize such traffic rules. Still, temporal logic rules often result in constraints that are hard to solve using optimization-based motion planners. Reinforcement learning (RL) is a promising method to find motion plans for autonomous vehicles. However, vanilla RL algorithms are based on random exploration and do not automatically comply with traffic rules. Our approach accomplishes guaranteed rule-compliance by integrating temporal logic specifications into RL. Specifically, we consider the application of vessels on the open sea, which must adhere to the Convention on the International Regulations for Preventing Collisions at Sea (COLREGS). To efficiently synthesize rule-compliant actions, we combine predicates based on set-based prediction with a statechart representing our formalized rules and their priorities. Action masking then restricts the RL agent to this set of verified rule-compliant actions. In numerical evaluations on critical maritime traffic situations, our agent always complies with the formalized legal rules and never collides while achieving a high goal-reaching rate during training and deployment. In contrast, vanilla and traffic rule-informed RL agents frequently violate traffic rules and collide even after training.

Related papers

• Driving with Regulation: Interpretable Decision-Making for Autonomous Vehicles with Retrieval-Augmented Reasoning via LLM [11.725133614445093]
  This work presents an interpretable decision-making framework for autonomous vehicles. We develop a Traffic Regulation Retrieval (TRR) Agent based on Retrieval-Augmented Generation (RAG) Given the semantic complexity of the retrieved rules, we also design a reasoning module powered by a Large Language Model (LLM)
  arXiv  Detail & Related papers  (2024-10-07T05:27:22Z)
• TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rules [0.0]
  TR2MTL is a framework that employs large language models (LLMs) to automatically translate traffic rules into metric temporal logic (MTL) It is envisioned as a human-in-loop system for AV rule formalization. It can be extended to various forms of temporal logic and rules.
  arXiv  Detail & Related papers  (2024-06-09T09:55:04Z)
• Learning Realistic Traffic Agents in Closed-loop [36.38063449192355]
  Reinforcement learning (RL) can train traffic agents to avoid infractions, but using RL alone results in unhuman-like driving behaviors. We propose Reinforcing Traffic Rules (RTR) to match expert demonstrations under a traffic compliance constraint. Our experiments show that RTR learns more realistic and generalizable traffic simulation policies.
  arXiv  Detail & Related papers  (2023-11-02T16:55:23Z)
• CAT: Closed-loop Adversarial Training for Safe End-to-End Driving [54.60865656161679]
  Adversarial Training (CAT) is a framework for safe end-to-end driving in autonomous vehicles. Cat aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios. Cat can effectively generate adversarial scenarios countering the agent being trained.
  arXiv  Detail & Related papers  (2023-10-19T02:49:31Z)
• Guided Conditional Diffusion for Controllable Traffic Simulation [42.198185904248994]
  Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic. We develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time.
  arXiv  Detail & Related papers  (2022-10-31T14:44:59Z)
• Quantification of Actual Road User Behavior on the Basis of Given Traffic Rules [4.731404257629232]
  We present an approach to derive the distribution of degrees of rule conformity from human driving data. We demonstrate our method with the Open Motion dataset and Safety Distance and Speed Limit rules.
  arXiv  Detail & Related papers  (2022-02-07T09:14:53Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)
• Emergent Road Rules In Multi-Agent Driving Environments [84.82583370858391]
  We analyze what ingredients in driving environments cause the emergence of road rules. We find that two crucial factors are noisy perception and agents' spatial density. Our results add empirical support for the social road rules that countries worldwide have agreed on for safe, efficient driving.
  arXiv  Detail & Related papers  (2020-11-21T09:43:50Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)
• Certified Reinforcement Learning with Logic Guidance [78.2286146954051]
  We propose a model-free RL algorithm that enables the use of Linear Temporal Logic (LTL) to formulate a goal for unknown continuous-state/action Markov Decision Processes (MDPs) The algorithm is guaranteed to synthesise a control policy whose traces satisfy the specification with maximal probability.
  arXiv  Detail & Related papers  (2019-02-02T20:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.