Critic-Free Deep Reinforcement Learning for Maritime Coverage Path Planning on Irregular Hexagonal Grids

• URL: http://arxiv.org/abs/2603.28385v1
• Date: Mon, 30 Mar 2026 12:56:38 GMT
• Title: Critic-Free Deep Reinforcement Learning for Maritime Coverage Path Planning on Irregular Hexagonal Grids
• Abstract summary: Maritime surveillance missions rely on the efficient allocation of sensing assets over vast and geometrically complex areas.<n>Traditional Coverage Path Planning approaches struggle with irregular coastlines, islands, and exclusion zones.<n>We propose a Deep Reinforcement Learning framework to solve CPP on hexagonal grid representations of irregular maritime areas.
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Maritime surveillance missions, such as search and rescue and environmental monitoring, rely on the efficient allocation of sensing assets over vast and geometrically complex areas. Traditional Coverage Path Planning (CPP) approaches depend on decomposition techniques that struggle with irregular coastlines, islands, and exclusion zones, or require computationally expensive re-planning for every instance. We propose a Deep Reinforcement Learning (DRL) framework to solve CPP on hexagonal grid representations of irregular maritime areas. Unlike conventional methods, we formulate the problem as a neural combinatorial optimization task where a Transformer-based pointer policy autoregressively constructs coverage tours. To overcome the instability of value estimation in long-horizon routing problems, we implement a critic-free Group-Relative Policy Optimization (GRPO) scheme. This method estimates advantages through within-instance comparisons of sampled trajectories rather than relying on a value function. Experiments on 1,000 unseen synthetic maritime environments demonstrate that a trained policy achieves a 99.0% Hamiltonian success rate, more than double the best heuristic (46.0%), while producing paths 7% shorter and with 24% fewer heading changes than the closest baseline. All three inference modes (greedy, stochastic sampling, and sampling with 2-opt refinement) operate under 50~ms per instance on a laptop GPU, confirming feasibility for real-time on-board deployment.

Related papers

• Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors [1.2246649738388389]
  We present a learning-based approach to kinodynamic online motion planning for an Unmanned Aerial Vehicle (UAV) operating at a fixed altitude in unknown dynamic environments.<n>The proposed framework integrates a behavior grid map representation with Deep Reinforcement Learning (DRL) for stable policy learning in continuous control.<n>The key idea is the co-design of a state representation and control policy that enables reliable navigation using only three low-cost directional range sensors.
  arXiv  Detail & Related papers  (2026-09-13T10:53:43Z)
• Hybrid Artificial Potential Fields and Spatio-Temporal Transformers for Real-Time AUV Path Planning [0.0]
  This paper presents a comparative evaluation of path planning algorithms for Autonomous Underwater Vehicles (AUVs)<n>Special emphasis is placed on a proposed hybrid approach combining Artificial Potential Fields (APF) with a Spatio-Temporal Transformer (ST) Transformer.<n>The Hybrid APF + ST-Transformer demonstrated superior balanced performance, achieving the shortest average path length (943.15 units), a low collision rate (0.031), and efficient computation time (0.96 s)
  arXiv  Detail & Related papers  (2026-07-27T20:34:33Z)
• Backpropagating Through Simulation: Analytic Policy Gradients for Sample and Learning Efficient Differentiable Continuous Control [0.0]
  Analytic Policy Gradients (APG) and PPO are evaluated against PPO on four continuous control tasks of increasing dynamical complexity.<n>We report a segmented backpropagation scheme with MC and critic-based bootstrap modes that mitigates gradient degradation on long-horizon tasks.
  arXiv  Detail & Related papers  (2026-06-19T15:22:51Z)
• Viewpoint-Agnostic Grasp Pipeline using VLM and Partial Observations [0.6263680699548958]
  We present an end-to-end pipeline for language-guided grasping that bridges open-vocabulary target selection to safe grasp execution on a real robot.<n>Given a natural-language command, the system grounds the target in RGB using open-vocabulary detection and promptable instance segmentation.<n>We then generate and collision-filter 6-DoF grasp candidates and select an executable grasp using safety-oriented grasps that account for reachability, approach feasibility, and clearance.
  arXiv  Detail & Related papers  (2026-03-09T00:42:32Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• Sequential Bayesian Optimal Experimental Design in Infinite Dimensions via Policy Gradient Reinforcement Learning [3.2580743227673694]
  High-fidelity approaches require repeated forward and adjoint PDE solves inside nested Bayesian inversion and design loops.<n>We formulate SBOED as a finite-horizon Markov decision process and learn an amortized design policy via policy-gradient reinforcement learning.<n> Numerical experiments on sequential multi-sensor placement for contaminant source tracking demonstrate approximately $100times$ speedup over high-fidelity finite element methods.
  arXiv  Detail & Related papers  (2026-01-09T15:44:49Z)
• Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach [0.6533458718563319]
  Off-policy methods promise high sample efficiency, a vital trait for minimizing costly and unsafe real-world fine-tuning.<n>On-policy methods often exhibit greater training stability, which is essential for reliable convergence in hazard-dense environments.<n>This work provides evidence that for high-precision, safety-critical navigation tasks, the reliable convergence of a well-established on-policy method can be more decisive than the nominal sample efficiency of an off-policy algorithm.
  arXiv  Detail & Related papers  (2025-08-22T21:29:59Z)
• Achieving $\widetilde{\mathcal{O}}(\sqrt{T})$ Regret in Average-Reward POMDPs with Known Observation Models [69.1820058966619]
  We tackle average-reward infinite-horizon POMDPs with an unknown transition model.<n>We present a novel and simple estimator that overcomes this barrier.
  arXiv  Detail & Related papers  (2025-01-30T22:29:41Z)
• Zonal RL-RRT: Integrated RL-RRT Path Planning with Collision Probability and Zone Connectivity [11.134855513221359]
  We introduce a novel path-planning algorithm, Zonal RL-RRT, that leverages kd-tree partitioning to segment the map into zones while addressing zone connectivity. Our algorithm achieves a 3x improvement in time efficiency compared to basic sampling methods such as RRT and RRT* in forest-like maps. Compared to learning-based methods like NeuralRRT* and MPNetSMP, as well as the RRT*J, our algorithm demonstrates, on average, 1.5x better performance in the same environments.
  arXiv  Detail & Related papers  (2024-10-31T17:57:51Z)
• Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation [72.24964965882783]
  Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error.<n>Real-world robotic tasks often suffer from sparse rewards, leading to inefficient exploration and suboptimal policies.<n>We introduce Confidence-Controlled Exploration (CCE), a novel method that improves sample efficiency in RL-based robotic navigation without modifying the reward function.
  arXiv  Detail & Related papers  (2023-06-09T18:45:15Z)
• Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees [56.848265937921354]
  Inverse reinforcement learning (IRL) aims to recover the reward function and the associated optimal policy. Many algorithms for IRL have an inherently nested structure. We develop a novel single-loop algorithm for IRL that does not compromise reward estimation accuracy.
  arXiv  Detail & Related papers  (2022-10-04T17:13:45Z)
• Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology [0.0]
  A general control policy framework based on deep reinforcement learning (DRL) is introduced for closed-loop decision making in subsurface flow settings. The DRL-based methodology is shown to result in an NPV increase of 15% (for the 2D cases) and 33% (3D cases) relative to robust optimization over prior models.
  arXiv  Detail & Related papers  (2022-03-24T22:50:47Z)
• Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policy [95.98698822755227]
  We make the first attempt to study risk-sensitive deep reinforcement learning under the average reward setting with the variance risk criteria. We propose an actor-critic algorithm that iteratively and efficiently updates the policy, the Lagrange multiplier, and the Fenchel dual variable.
  arXiv  Detail & Related papers  (2020-12-28T05:02:26Z)
• Deep Reinforcement Learning with Robust and Smooth Policy [90.78795857181727]
  We propose to learn a smooth policy that behaves smoothly with respect to states. We develop a new framework -- textbfSmooth textbfRegularized textbfReinforcement textbfLearning ($textbfSR2textbfL$), where the policy is trained with smoothness-inducing regularization. Such regularization effectively constrains the search space, and enforces smoothness in the learned policy.
  arXiv  Detail & Related papers  (2020-03-21T00:10:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.