2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

• URL: http://arxiv.org/abs/2307.16769v3
• Date: Wed, 21 Aug 2024 12:19:12 GMT
• Title: 2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following
• Authors: Martin Waltz, Niklas Paulig, Ostap Okhrin,
• Abstract summary: This paper proposes a realistic modularized framework for controlling autonomous surface vehicles (ASVs) on inland waterways (IWs) based on deep reinforcement learning (DRL) The framework improves operational safety and comprises two levels: a high-level local path planning (LPP) unit and a low-level path following (PF) unit, each consisting of a DRL agent.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a realistic modularized framework for controlling autonomous surface vehicles (ASVs) on inland waterways (IWs) based on deep reinforcement learning (DRL). The framework improves operational safety and comprises two levels: a high-level local path planning (LPP) unit and a low-level path following (PF) unit, each consisting of a DRL agent. The LPP agent is responsible for planning a path under consideration of dynamic vessels, closing a gap in the current research landscape. In addition, the LPP agent adequately considers traffic rules and the geometry of the waterway. We thereby introduce a novel application of a spatial-temporal recurrent neural network architecture to continuous action spaces. The LPP agent outperforms a state-of-the-art artificial potential field (APF) method by increasing the minimum distance to other vessels by 65% on average. The PF agent performs low-level actuator control while accounting for shallow water influences and the environmental forces winds, waves, and currents. Compared with a proportional-integral-derivative (PID) controller, the PF agent yields only 61% of the mean cross-track error (MCTE) while significantly reducing control effort (CE) in terms of the required absolute rudder angle. Lastly, both agents are jointly validated in simulation, employing the lower Elbe in northern Germany as an example case and using real automatic identification system (AIS) trajectories to model the behavior of other ships.

Related papers

• Navigation in a simplified Urban Flow through Deep Reinforcement Learning [0.9217021281095907]
  Unmanned aerial vehicles (UAVs) in urban environments require a strategy to minimize their environmental impact. Our goal is to develop DRL algorithms capable of enabling the autonomous navigation of UAVs in urban environments.
  arXiv  Detail & Related papers  (2024-09-26T15:05:15Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Autonomous Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving. Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner. Experiments conducted on nuScenes dataset demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• AD-H: Autonomous Driving with Hierarchical Agents [64.49185157446297]
  We propose to connect high-level instructions and low-level control signals with mid-level language-driven commands. We implement this idea through a hierarchical multi-agent driving system named AD-H.
  arXiv  Detail & Related papers  (2024-06-05T17:25:46Z)
• Safety Aware Autonomous Path Planning Using Model Predictive Reinforcement Learning for Inland Waterways [2.0623470039259946]
  We propose a novel path planning approach based on reinforcement learning called Model Predictive Reinforcement Learning (MPRL) MPRL calculates a series of waypoints for the vessel to follow. We demonstrate our approach on two scenarios and compare the resulting path with path planning using a Frenet frame and path planning based on a proximal policy optimization (PPO) agent.
  arXiv  Detail & Related papers  (2023-11-16T13:12:58Z)
• ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments [56.194988818341976]
  Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. We propose ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets.
  arXiv  Detail & Related papers  (2023-04-06T13:07:17Z)
• Robust Path Following on Rivers Using Bootstrapped Reinforcement Learning [0.0]
  This paper develops a Deep Reinforcement Learning (DRL)-agent for navigation and control of autonomous surface vessels (ASV) on inland waterways. A state-of-the-art bootstrapped Q-learning algorithm in combination with a versatile training environment generator leads to a robust and accurate rudder controller.
  arXiv  Detail & Related papers  (2023-03-24T07:21:27Z)
• SEA: Bridging the Gap Between One- and Two-stage Detector Distillation via SEmantic-aware Alignment [76.80165589520385]
  We name our method SEA (SEmantic-aware Alignment) distillation given the nature of abstracting dense fine-grained information. It achieves new state-of-the-art results on the challenging object detection task on both one- and two-stage detectors.
  arXiv  Detail & Related papers  (2022-03-02T04:24:05Z)
• Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning [1.304892050913381]
  Deep reinforcement learning (DRL) has shown great potential for a wide range of applications. In this work, a subset of the International Regulations for Preventing Collisions at Sea (COLREGs) is incorporated into a DRL-based path following and obstacle avoidance system. The resulting autonomous agent dynamically interpolates between path following and COLREG-compliant collision avoidance in the training scenario, isolated encounter situations, and AIS-based simulations of real-world scenarios.
  arXiv  Detail & Related papers  (2021-11-30T21:32:59Z)
• Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning [21.500697097095408]
  Planning safe trajectories under uncertain and dynamic conditions makes the autonomous driving problem significantly complex. Current sampling-based methods such as Rapidly Exploring Random Trees (RRTs) are not ideal for this problem because of the high computational cost. We propose a Hierarchical Reinforcement Learning structure combined with a Proportional-Integral-Derivative (PID) controller for trajectory planning.
  arXiv  Detail & Related papers  (2020-11-09T20:49:54Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)
• Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous [66.6895109554163]
  Underlying the human ability to align goals with other agents is their ability to predict the intentions of others and actively update their own plans. We propose hierarchical predictive planning (HPP), a model-based reinforcement learning method for decentralized multiagent rendezvous.
  arXiv  Detail & Related papers  (2020-03-15T19:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.