Depth-Constrained ASV Navigation with Deep RL and Limited Sensing

• URL: http://arxiv.org/abs/2504.18253v1
• Date: Fri, 25 Apr 2025 10:56:56 GMT
• Title: Depth-Constrained ASV Navigation with Deep RL and Limited Sensing
• Authors: Amirhossein Zhalehmehrabi, Daniele Meli, Francesco Dal Santo, Francesco Trotti, Alessandro Farinelli,
• Abstract summary: We propose a reinforcement learning framework for ASV navigation under depth constraints.<n>To enhance environmental awareness, we integrate GP regression into the RL framework.<n>We demonstrate effective sim-to-real transfer, ensuring that trained policies generalize well to real-world aquatic conditions.
• Score: 45.77464360746532
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous Surface Vehicles (ASVs) play a crucial role in maritime operations, yet their navigation in shallow-water environments remains challenging due to dynamic disturbances and depth constraints. Traditional navigation strategies struggle with limited sensor information, making safe and efficient operation difficult. In this paper, we propose a reinforcement learning (RL) framework for ASV navigation under depth constraints, where the vehicle must reach a target while avoiding unsafe areas with only a single depth measurement per timestep from a downward-facing Single Beam Echosounder (SBES). To enhance environmental awareness, we integrate Gaussian Process (GP) regression into the RL framework, enabling the agent to progressively estimate a bathymetric depth map from sparse sonar readings. This approach improves decision-making by providing a richer representation of the environment. Furthermore, we demonstrate effective sim-to-real transfer, ensuring that trained policies generalize well to real-world aquatic conditions. Experimental results validate our method's capability to improve ASV navigation performance while maintaining safety in challenging shallow-water environments.

Related papers

• Vision-Based Deep Reinforcement Learning of UAV Autonomous Navigation Using Privileged Information [6.371251946803415]
  DPRL is an end-to-end policy designed to address the challenge of high-speed autonomous UAV navigation under partially observable environmental conditions.<n>We leverage an asymmetric Actor-Critic architecture to provide the agent with privileged information during training.<n>We conduct extensive simulations across various scenarios, benchmarking our DPRL algorithm against the state-of-the-art navigation algorithms.
  arXiv  Detail & Related papers  (2024-12-09T09:05:52Z)
• Evaluating Robustness of Reinforcement Learning Algorithms for Autonomous Shipping [2.9109581496560044]
  This paper examines the robustness of benchmark deep reinforcement learning (RL) algorithms, implemented for inland waterway transport (IWT) within an autonomous shipping simulator. We show that a model-free approach can achieve an adequate policy in the simulator, successfully navigating port environments never encountered during training.
  arXiv  Detail & Related papers  (2024-11-07T17:55:07Z)
• Deep-Sea A*+: An Advanced Path Planning Method Integrating Enhanced A* and Dynamic Window Approach for Autonomous Underwater Vehicles [1.3807821497779342]
  Extreme conditions in the deep-sea environment pose significant challenges for underwater operations. We propose an advanced path planning methodology that integrates an improved A* algorithm with the Dynamic Window Approach (DWA) Our proposed method surpasses the traditional A* algorithm in terms of path smoothness, obstacle avoidance, and real-time performance.
  arXiv  Detail & Related papers  (2024-10-22T07:29:05Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• EnCoMP: Enhanced Covert Maneuver Planning with Adaptive Threat-Aware Visibility Estimation using Offline Reinforcement Learning [0.6597195879147555]
  We propose EnCoMP, an enhanced navigation framework to enable robots to navigate covertly in diverse outdoor settings. We generate high-fidelity multi-map representations, including cover maps, potential threat maps, height maps, and goal maps from LiDAR point clouds. We demonstrate our method's capabilities on a physical Jackal robot, showing extensive experiments across diverse terrains.
  arXiv  Detail & Related papers  (2024-03-29T07:03:10Z)
• VAPOR: Legged Robot Navigation in Outdoor Vegetation Using Offline Reinforcement Learning [53.13393315664145]
  We present VAPOR, a novel method for autonomous legged robot navigation in unstructured, densely vegetated outdoor environments. Our method trains a novel RL policy using an actor-critic network and arbitrary data collected in real outdoor vegetation. We observe that VAPOR's actions improve success rates by up to 40%, decrease the average current consumption by up to 2.9%, and decrease the normalized trajectory length by up to 11.2%.
  arXiv  Detail & Related papers  (2023-09-14T16:21:27Z)
• Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection [0.0]
  navigational autonomy in unmanned surface vehicles (USVs) in the maritime sector can lead to safer waters as well as reduced operating costs. We describe the novel development of a COLREG-compliant DRL-based collision avoidant navigational system with CV-based awareness in a realistic ocean simulation environment.
  arXiv  Detail & Related papers  (2023-07-13T11:20:18Z)
• Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation [72.24964965882783]
  Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error.<n>Real-world robotic tasks often suffer from sparse rewards, leading to inefficient exploration and suboptimal policies.<n>We introduce Confidence-Controlled Exploration (CCE), a novel method that improves sample efficiency in RL-based robotic navigation without modifying the reward function.
  arXiv  Detail & Related papers  (2023-06-09T18:45:15Z)
• ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments [56.194988818341976]
  Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. We propose ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets.
  arXiv  Detail & Related papers  (2023-04-06T13:07:17Z)
• Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation [78.17108227614928]
  We propose a benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation. We consider a value-based and policy-gradient Deep Reinforcement Learning (DRL) We also propose a verification strategy that checks the behavior of the trained models over a set of desired properties.
  arXiv  Detail & Related papers  (2021-12-16T16:53:56Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.