Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning

• URL: http://arxiv.org/abs/2112.00115v1
• Date: Tue, 30 Nov 2021 21:32:59 GMT
• Title: Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning
• Authors: Thomas Nakken Larsen, Amalie Heiberg, Eivind Meyer, Adil Rasheeda, Omer San, Damiano Varagnolo
• Abstract summary: Deep reinforcement learning (DRL) has shown great potential for a wide range of applications. In this work, a subset of the International Regulations for Preventing Collisions at Sea (COLREGs) is incorporated into a DRL-based path following and obstacle avoidance system. The resulting autonomous agent dynamically interpolates between path following and COLREG-compliant collision avoidance in the training scenario, isolated encounter situations, and AIS-based simulations of real-world scenarios.
• Score: 1.304892050913381
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous systems are becoming ubiquitous and gaining momentum within the marine sector. Since the electrification of transport is happening simultaneously, autonomous marine vessels can reduce environmental impact, lower costs, and increase efficiency. Although close monitoring is still required to ensure safety, the ultimate goal is full autonomy. One major milestone is to develop a control system that is versatile enough to handle any weather and encounter that is also robust and reliable. Additionally, the control system must adhere to the International Regulations for Preventing Collisions at Sea (COLREGs) for successful interaction with human sailors. Since the COLREGs were written for the human mind to interpret, they are written in ambiguous prose and therefore not machine-readable or verifiable. Due to these challenges and the wide variety of situations to be tackled, classical model-based approaches prove complicated to implement and computationally heavy. Within machine learning (ML), deep reinforcement learning (DRL) has shown great potential for a wide range of applications. The model-free and self-learning properties of DRL make it a promising candidate for autonomous vessels. In this work, a subset of the COLREGs is incorporated into a DRL-based path following and obstacle avoidance system using collision risk theory. The resulting autonomous agent dynamically interpolates between path following and COLREG-compliant collision avoidance in the training scenario, isolated encounter situations, and AIS-based simulations of real-world scenarios.

Related papers

• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable [88.08120417169971]
  Machine learning based autonomous driving systems often face challenges with safety-critical scenarios that are rare in real-world data. This work explores generating safety-critical driving scenarios by modifying complex real-world regular scenarios through trajectory optimization. Our approach addresses unrealistic diverging trajectories and unavoidable collision scenarios that are not useful for training robust planner.
  arXiv  Detail & Related papers  (2024-09-12T08:26:33Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• Variational Autoencoders for exteroceptive perception in reinforcement learning-based collision avoidance [0.0]
  Deep Reinforcement Learning (DRL) has emerged as a promising control framework. Current DRL algorithms require disproportionally large computational resources to find near-optimal policies. This paper presents a comprehensive exploration of our proposed approach in maritime control systems.
  arXiv  Detail & Related papers  (2024-03-31T09:25:28Z)
• Spacecraft Autonomous Decision-Planning for Collision Avoidance: a Reinforcement Learning Approach [0.0]
  This work proposes an implementation of autonomous CA decision-making capabilities on spacecraft based on Reinforcement Learning techniques. The proposed framework considers imperfect monitoring information about the status of the debris in orbit and allows the AI system to effectively learn policies to perform accurate Collision Avoidance Maneuvers (CAMs) The objective is to successfully delegate the decision-making process for autonomously implementing a CAM to the spacecraft without human intervention.
  arXiv  Detail & Related papers  (2023-10-29T10:15:33Z)
• Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection [0.0]
  navigational autonomy in unmanned surface vehicles (USVs) in the maritime sector can lead to safer waters as well as reduced operating costs. We describe the novel development of a COLREG-compliant DRL-based collision avoidant navigational system with CV-based awareness in a realistic ocean simulation environment.
  arXiv  Detail & Related papers  (2023-07-13T11:20:18Z)
• Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
  We propose Iso-Dream++, a model-based reinforcement learning approach. We perform policy optimization based on the decoupled latent imaginations. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
  arXiv  Detail & Related papers  (2023-03-27T02:55:56Z)
• Learned Risk Metric Maps for Kinodynamic Systems [54.49871675894546]
  We present Learned Risk Metric Maps for real-time estimation of coherent risk metrics of high dimensional dynamical systems. LRMM models are simple to design and train, requiring only procedural generation of obstacle sets, state and control sampling, and supervised training of a function approximator.
  arXiv  Detail & Related papers  (2023-02-28T17:51:43Z)
• Isolating and Leveraging Controllable and Noncontrollable Visual Dynamics in World Models [65.97707691164558]
  We present Iso-Dream, which improves the Dream-to-Control framework in two aspects. First, by optimizing inverse dynamics, we encourage world model to learn controllable and noncontrollable sources. Second, we optimize the behavior of the agent on the decoupled latent imaginations of the world model.
  arXiv  Detail & Related papers  (2022-05-27T08:07:39Z)
• Deep Reinforcement Learning Controller for 3D Path-following and Collision Avoidance by Autonomous Underwater Vehicles [0.0]
  In complex systems, such as autonomous underwater vehicles, decision making becomes non-trivial. We propose a solution using state-of-the-art Deep Reinforcement Learning (DRL) techniques. Our results demonstrate the viability of DRL in path-following and avoiding collisions toward achieving human-level decision making in autonomous vehicle systems.
  arXiv  Detail & Related papers  (2020-06-17T11:54:53Z)
• COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learning [0.0]
  Path Following and Collision Avoidance, be it for unmanned surface vessels or other autonomous vehicles, are two fundamental guidance problems in robotics. In this article, we explore the potential of Proximal Policy Optimization (PPO), a DRL algorithm with demonstrated state-of-the-art performance on Continuous Control tasks. Based on high-fidelity elevation and AIS tracking data from the Trondheim Fjord, an inlet of the Norwegian sea, we evaluate the trained agent's performance in challenging, dynamic real-world scenarios.
  arXiv  Detail & Related papers  (2020-06-16T22:05:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.