Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learning

• URL: http://arxiv.org/abs/2607.13553v1
• Date: Wed, 15 Jul 2026 07:57:26 GMT
• Title: Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learning
• Abstract summary: We present a reinforcement learning approach to train autonomous agents to reach arbitrary targets in a chaotic double-gyre flow.<n>We analyze the impact of providing agents with explicit global flow parameters.<n>The experiments reveal a trade-off in sensor utility: velocity-aware agents optimize energy efficiency, whereas vorticity sensors provide superior structural mapping and achieve better target proximity.
• Score: 0.815557531820863
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous robotic navigation in nonstationary time-varying fluid flows remains a fundamental challenge due to partial observability and the unpredictability of realistic environments. While classical optimal control frameworks employed in robotics require unrealistic a-priori global flow knowledge, biological systems are able to navigate successfully by exploiting localized sensory cues. In this work we present a reinforcement learning approach using the TD3 algorithm to train autonomous agents to reach arbitrary targets within a parametric, chaotic double-gyre flow. To investigate optimal sensory mechanisms, we evaluate five bio-inspired observation strategies based on relative position, local velocity or local vorticity measures, and short-term memory variants. Additionally, we analyze the impact of providing agents with explicit global flow parameters. Numerical results demonstrate that an agent that is able to sense and remember a set number of flow velocity measures achieves the highest performance. The experiments reveal a trade-off in sensor utility: velocity-aware agents optimize energy efficiency, whereas vorticity sensors provide superior structural mapping and achieve better target proximity. Incorporating explicit global flow parameters is shown to decrease navigation performance. This behavior suggests that reinforcement learning-based autonomous systems develop more robust and general policies when restricted to implicit flow representations. The presented results offer insights for improving the transition of bio-inspired robotic navigation from simulation to real-world environments.

Related papers

• MORN: Metacognitive Object-Goal Regulation for Resource-Rational Long-Horizon Navigation [8.894907353056125]
  We present MORN, an executive architecture inspired by Dual-Process Theory in cognitive science.<n>By formalizing three neuro-cognitive states, Potentiality Index, Persistence Gating, and Evidence Accumulation, MORN dynamically regulates the mission schedule.<n>This mechanism effectively neutralizes the Sunk Cost Fallacy, enabling agents to abort zombie goals early and decisively commit to achievable ones.
  arXiv  Detail & Related papers  (2026-05-16T10:59:51Z)
• NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation [9.168410028117089]
  We present a complete training and deployment pipeline for reinforcement learning-based navigation.<n>We disentangle the key factors affecting sim-to-real transfer in reinforcement learning-based navigation, including sensor noise, perception failures, system latency, and control response.<n>We propose perturbation-aware fine-tuning, a post-training adaptation strategy that improves transfer robustness.
  arXiv  Detail & Related papers  (2026-05-15T02:58:25Z)
• Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation [63.11032720821731]
  textitSAGE is a framework that enables agents to learn within a physics-grounded semantic abstraction rather than a photorealistic simulation.<n>We demonstrate that textitSAGE significantly improves planner-assisted embodied navigation, achieving a 53.21% LLM-Match Success Rate on A-EQA.
  arXiv  Detail & Related papers  (2026-05-11T07:34:30Z)
• IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigation [56.43007596544299]
  IndustryNav is the first dynamic industrial navigation benchmark for active spatial reasoning.<n>A study of nine state-of-the-art Visual Large Language Models reveals that closed-source models maintain a consistent advantage.
  arXiv  Detail & Related papers  (2025-11-21T16:48:49Z)
• DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation [55.888688171010365]
  DORAEMON is a cognitive-inspired framework consisting of Ventral and Dorsal Streams that mimics human navigation capabilities.<n>We evaluate DORAEMON on the HM3D, MP3D and GOAT datasets, where it achieves state-of-the-art performance on both success rate (SR) and success weighted by path length (SPL) metrics.
  arXiv  Detail & Related papers  (2025-05-28T04:46:13Z)
• A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows [0.0]
  Navigating in a fluid flow while being carried by it, using only accessible information from on-board sensors, is a problem commonly faced by small planktonic organisms.<n>In the last ten years, the fluid mechanics community has widely adopted reinforcement learning, often in the form of its simplest implementations.<n>But it is unclear how good are the strategies learned by these algorithms.
  arXiv  Detail & Related papers  (2025-05-08T09:17:26Z)
• NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistants [24.689242976554482]
  Navigating unfamiliar environments presents significant challenges for household robots.<n>Existing reinforcement learning methods cannot be directly transferred to new environments.<n>We try to transfer the logical knowledge and the generalization ability of pre-trained foundation models to zero-shot navigation.
  arXiv  Detail & Related papers  (2025-02-19T17:27:47Z)
• Understanding Optimization in Deep Learning with Central Flows [95.5647720254338]
  We develop theory that can describe the dynamics of optimization in a complex regime.<n>Our results suggest that central flows can be a valuable theoretical tool for reasoning about optimization in deep learning.
  arXiv  Detail & Related papers  (2024-10-31T17:58:13Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation [73.78984332354636]
  CorNav is a novel zero-shot framework for vision-and-language navigation. It incorporates environmental feedback for refining future plans and adjusting its actions. It consistently outperforms all baselines in a zero-shot multi-task setting.
  arXiv  Detail & Related papers  (2023-06-17T11:44:04Z)
• StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
  We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception. We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy. Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
  arXiv  Detail & Related papers  (2022-07-21T12:03:02Z)
• Learning Efficient Navigation in Vortical Flow Fields [6.585044528359311]
  We apply a novel Reinforcement Learning algorithm to steer a fixed-speed swimmer through an unsteady two-dimensional flow field. The algorithm entails inputting environmental cues into a deep neural network that determines the swimmer's actions. A velocity sensing approach outperformed a bio-mimetic vorticity sensing approach by nearly two-fold in success rate.
  arXiv  Detail & Related papers  (2021-02-21T07:25:03Z)
• Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents [90.65480527538723]
  We develop point-goal navigation agents that rely on visual estimates of egomotion under noisy action dynamics. Our agent was the runner-up in the PointNav track of CVPR 2020 Habitat Challenge.
  arXiv  Detail & Related papers  (2020-09-07T16:52:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.