Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning

• URL: http://arxiv.org/abs/2103.00329v1
• Date: Sat, 27 Feb 2021 21:31:18 GMT
• Title: Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning
• Authors: Michele Buzzicotti, Luca Biferale, Fabio Bonaccorso, Patricio Clark di Leoni and Kristian Gustavsson
• Abstract summary: We present theoretical and numerical results concerning the problem to find the path that minimizes time to navigate between two given points in a complex fluid. We show that ActorCritic algorithms are able to find quasi-optimal solutions in the presence of either time-independent or chaotically evolving flow configurations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present theoretical and numerical results concerning the problem to find the path that minimizes the time to navigate between two given points in a complex fluid under realistic navigation constraints. We contrast deterministic Optimal Navigation (ON) control with stochastic policies obtained by Reinforcement Learning (RL) algorithms. We show that Actor-Critic RL algorithms are able to find quasi-optimal solutions in the presence of either time-independent or chaotically evolving flow configurations. For our application, ON solutions develop unstable behavior within the typical duration of the navigation process, and are therefore not useful in practice. We first explore navigation of turbulent flow using a constant propulsion speed. Based on a discretized phase-space, the propulsion direction is adjusted with the aim to minimize the time spent to reach the target. Further, we explore a case where additional control is obtained by allowing the engine to power off. Exploiting advection of the underlying flow, allows the target to be reached with less energy consumption. In this case, we optimize a linear combination between the total navigation time and the total time the engine is switched off. Our approach can be generalized to other setups, for example, navigation under imperfect environmental forecast or with different models for the moving vessel.

Related papers

• Understanding Optimization in Deep Learning with Central Flows [53.66160508990508]
  We show that an RMS's implicit behavior can be explicitly captured by a "central flow:" a differential equation. We show that these flows can empirically predict long-term optimization trajectories of generic neural networks.
  arXiv  Detail & Related papers  (2024-10-31T17:58:13Z)
• Partial End-to-end Reinforcement Learning for Robustness Against Modelling Error in Autonomous Racing [0.0]
  This paper addresses the issue of increasing the performance of reinforcement learning (RL) solutions for autonomous racing cars. We propose a partial end-to-end algorithm that decouples the planning and control tasks. By leveraging the robustness of a classical controller, our partial end-to-end driving algorithm exhibits better robustness towards model mismatches than standard end-to-end algorithms.
  arXiv  Detail & Related papers  (2023-12-11T14:27:10Z)
• AccFlow: Backward Accumulation for Long-Range Optical Flow [70.4251045372285]
  This paper proposes a novel recurrent framework called AccFlow for long-range optical flow estimation. We demonstrate the superiority of backward accumulation over conventional forward accumulation. Experiments validate the effectiveness of AccFlow in handling long-range optical flow estimation.
  arXiv  Detail & Related papers  (2023-08-25T01:51:26Z)
• ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments [56.194988818341976]
  Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. We propose ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets.
  arXiv  Detail & Related papers  (2023-04-06T13:07:17Z)
• Robust Path Following on Rivers Using Bootstrapped Reinforcement Learning [0.0]
  This paper develops a Deep Reinforcement Learning (DRL)-agent for navigation and control of autonomous surface vessels (ASV) on inland waterways. A state-of-the-art bootstrapped Q-learning algorithm in combination with a versatile training environment generator leads to a robust and accurate rudder controller.
  arXiv  Detail & Related papers  (2023-03-24T07:21:27Z)
• Globally Optimal Event-Based Divergence Estimation for Ventral Landing [55.29096494880328]
  Event sensing is a major component in bio-inspired flight guidance and control systems. We explore the usage of event cameras for predicting time-to-contact with the surface during ventral landing. This is achieved by estimating divergence (inverse TTC), which is the rate of radial optic flow, from the event stream generated during landing. Our core contributions are a novel contrast maximisation formulation for event-based divergence estimation, and a branch-and-bound algorithm to exactly maximise contrast and find the optimal divergence value.
  arXiv  Detail & Related papers  (2022-09-27T06:00:52Z)
• Time-Optimal Planning for Quadrotor Waypoint Flight [50.016821506107455]
  Planning time-optimal trajectories at the actuation limit of a quadrotor is an open problem. We propose a solution while exploiting the full quadrotor's actuator potential. We validate our method in real-world flights in one of the world's largest motion-capture systems.
  arXiv  Detail & Related papers  (2021-08-10T09:26:43Z)
• Learning Efficient Navigation in Vortical Flow Fields [6.585044528359311]
  We apply a novel Reinforcement Learning algorithm to steer a fixed-speed swimmer through an unsteady two-dimensional flow field. The algorithm entails inputting environmental cues into a deep neural network that determines the swimmer's actions. A velocity sensing approach outperformed a bio-mimetic vorticity sensing approach by nearly two-fold in success rate.
  arXiv  Detail & Related papers  (2021-02-21T07:25:03Z)
• Congestion-aware Evacuation Routing using Augmented Reality Devices [96.68280427555808]
  We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes among multiple destinations. A population density map, obtained on-the-fly by aggregating locations of evacuees from user-end Augmented Reality (AR) devices, is used to model the congestion distribution inside a building.
  arXiv  Detail & Related papers  (2020-04-25T22:54:35Z)
• Reinforcement Meta-Learning for Interception of Maneuvering Exoatmospheric Targets with Parasitic Attitude Loop [1.7663909228482466]
  We use Reinforcement Meta-Learning to optimize an adaptive integrated guidance, navigation, and control system suitable for exoatmospheric interception of a maneuvering target. The system maps observations consisting of strapdown seeker angles and rate gyro measurements directly to thruster on-off commands. We demonstrate that the optimized policy can adapt to parasitic effects including seeker angle measurement lag, thruster control lag, the parasitic attitude loop resulting from scale factor errors and Gaussian noise on angle and rotational velocity measurements.
  arXiv  Detail & Related papers  (2020-04-18T21:20:59Z)
• Enhance the performance of navigation: A two-stage machine learning approach [13.674463804942837]
  Real time traffic navigation is an important capability in smart transportation technologies. In this paper, we adopt the ideas of ensemble learning and develop a two-stage machine learning model to give accurate navigation results.
  arXiv  Detail & Related papers  (2020-04-02T08:55:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.