Navigation in a simplified Urban Flow through Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2409.17922v1
• Date: Thu, 26 Sep 2024 15:05:15 GMT
• Title: Navigation in a simplified Urban Flow through Deep Reinforcement Learning
• Authors: Federica Tonti, Jean Rabault, Ricardo Vinuesa,
• Abstract summary: Unmanned aerial vehicles (UAVs) in urban environments require a strategy to minimize their environmental impact. Our goal is to develop DRL algorithms capable of enabling the autonomous navigation of UAVs in urban environments.
• Score: 0.9217021281095907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing number of unmanned aerial vehicles (UAVs) in urban environments requires a strategy to minimize their environmental impact, both in terms of energy efficiency and noise reduction. In order to reduce these concerns, novel strategies for developing prediction models and optimization of flight planning, for instance through deep reinforcement learning (DRL), are needed. Our goal is to develop DRL algorithms capable of enabling the autonomous navigation of UAVs in urban environments, taking into account the presence of buildings and other UAVs, optimizing the trajectories in order to reduce both energetic consumption and noise. This is achieved using fluid-flow simulations which represent the environment in which UAVs navigate and training the UAV as an agent interacting with an urban environment. In this work, we consider a domain domain represented by a two-dimensional flow field with obstacles, ideally representing buildings, extracted from a three-dimensional high-fidelity numerical simulation. The presented methodology, using PPO+LSTM cells, was validated by reproducing a simple but fundamental problem in navigation, namely the Zermelo's problem, which deals with a vessel navigating in a turbulent flow, travelling from a starting point to a target location, optimizing the trajectory. The current method shows a significant improvement with respect to both a simple PPO and a TD3 algorithm, with a success rate (SR) of the PPO+LSTM trained policy of 98.7%, and a crash rate (CR) of 0.1%, outperforming both PPO (SR = 75.6%, CR=18.6%) and TD3 (SR=77.4% and CR=14.5%). This is the first step towards DRL strategies which will guide UAVs in a three-dimensional flow field using real-time signals, making the navigation efficient in terms of flight time and avoiding damages to the vehicle.

Related papers

• Guidance Design for Escape Flight Vehicle Using Evolution Strategy Enhanced Deep Reinforcement Learning [6.037202026682975]
  We consider the scenario where the escape flight vehicle (EFV) generates guidance commands based on DRL and the pursuit flight vehicle (PFV) generates guidance commands based on the proportional navigation method. For the EFV, the objective of the guidance design entails progressively maximizing the residual velocity, subject to the constraint imposed by the given evasion distance. In the first step, we use the proximal policy optimization (PPO) algorithm to generate the guidance commands of the EFV. In the second step, we propose to invoke the evolution strategy (ES) based algorithm, which uses the result of PPO as the
  arXiv  Detail & Related papers  (2024-05-04T06:18:15Z)
• Meta Reinforcement Learning for Strategic IoT Deployments Coverage in Disaster-Response UAV Swarms [5.57865728456594]
  Unmanned Aerial Vehicles (UAVs) have grabbed the attention of researchers in academia and industry for their potential use in critical emergency applications. These applications include providing wireless services to ground users and collecting data from areas affected by disasters. UAVs' limited resources, energy budget, and strict mission completion time have posed challenges in adopting UAVs for these applications.
  arXiv  Detail & Related papers  (2024-01-20T05:05:39Z)
• VAPOR: Legged Robot Navigation in Outdoor Vegetation Using Offline Reinforcement Learning [53.13393315664145]
  We present VAPOR, a novel method for autonomous legged robot navigation in unstructured, densely vegetated outdoor environments. Our method trains a novel RL policy using an actor-critic network and arbitrary data collected in real outdoor vegetation. We observe that VAPOR's actions improve success rates by up to 40%, decrease the average current consumption by up to 2.9%, and decrease the normalized trajectory length by up to 11.2%.
  arXiv  Detail & Related papers  (2023-09-14T16:21:27Z)
• Trajectory Design for UAV-Based Internet-of-Things Data Collection: A Deep Reinforcement Learning Approach [93.67588414950656]
  In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a 3D environment. We present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm. Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.
  arXiv  Detail & Related papers  (2021-07-23T03:33:29Z)
• Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks [151.27147513363502]
  This paper studies the problem of the trajectory design for a group of energyconstrained drones operating in dynamic wireless network environments. A value based reinforcement learning (VDRL) solution and a metatraining mechanism is proposed.
  arXiv  Detail & Related papers  (2020-12-06T01:30:12Z)
• Congestion-aware Evacuation Routing using Augmented Reality Devices [96.68280427555808]
  We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes among multiple destinations. A population density map, obtained on-the-fly by aggregating locations of evacuees from user-end Augmented Reality (AR) devices, is used to model the congestion distribution inside a building.
  arXiv  Detail & Related papers  (2020-04-25T22:54:35Z)
• Learning in the Sky: An Efficient 3D Placement of UAVs [0.8399688944263842]
  We propose a learning-based mechanism for the three-dimensional deployment of UAVs assisting terrestrial cellular networks in the downlink. The problem is modeled as a non-cooperative game among UAVs in satisfaction form. To solve the game, we utilize a low complexity algorithm, in which unsatisfied UAVs update their locations based on a learning algorithm.
  arXiv  Detail & Related papers  (2020-03-02T15:16:00Z)
• Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach [88.45509934702913]
  We design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed. We incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS. By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time.
  arXiv  Detail & Related papers  (2020-02-21T07:29:15Z)
• Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms [98.78553146823829]
  Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks. In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
  arXiv  Detail & Related papers  (2020-02-19T14:04:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.