Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2510.25679v1
• Date: Wed, 29 Oct 2025 16:46:00 GMT
• Title: Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning
• Authors: Federica Tonti, Ricardo Vinuesa,
• Abstract summary: Unmanned Aerial Vehicles (UAVs) are increasingly populating urban areas for delivery and surveillance purposes.<n>In this work, we develop an optimal navigation strategy based on Deep Reinforcement Learning.<n>Results show a significant increase in the success rate (SR) and a lower crash rate (CR) compared to a PPO+LSTM, PPO+GTrXL and the classical Zermelo's navigation algorithm.
• Score: 3.306815791933257
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unmanned Aerial Vehicles (UAVs) are increasingly populating urban areas for delivery and surveillance purposes. In this work, we develop an optimal navigation strategy based on Deep Reinforcement Learning. The environment is represented by a three-dimensional high-fidelity simulation of an urban flow, characterized by turbulence and recirculation zones. The algorithm presented here is a flow-aware Proximal Policy Optimization (PPO) combined with a Gated Transformer eXtra Large (GTrXL) architecture, giving the agent richer information about the turbulent flow field in which it navigates. The results are compared with a PPO+GTrXL without the secondary prediction tasks, a PPO combined with Long Short Term Memory (LSTM) cells and a traditional navigation algorithm. The obtained results show a significant increase in the success rate (SR) and a lower crash rate (CR) compared to a PPO+LSTM, PPO+GTrXL and the classical Zermelo's navigation algorithm, paving the way to a completely reimagined UAV landscape in complex urban environments.

Related papers

• UrbanPulse: A Cross-City Deep Learning Framework for Ultra-Fine-Grained Population Transfer Prediction [0.0]
  UrbanPulse is a scalable deep learning framework that delivers ultrafine-grained, city-wide OD flow predictions.<n>It employs a three-stage learning strategy: pretraining on large-scale urban graphs, cold-start adaptation, and reinforcement learning fine-tuning.<n>It achieves state-of-the-art accuracy and scalability.
  arXiv  Detail & Related papers  (2025-07-23T20:44:25Z)
• FlexDrive: Toward Trajectory Flexibility in Driving Scene Reconstruction and Rendering [79.39246982782717]
  We introduce an Inverse View Warping technique to create compact and high-quality images as supervision for the reconstruction of the out-of-path views.<n>Our method achieves superior in-path and out-of-path reconstruction and rendering performance on the widely used Open dataset.
  arXiv  Detail & Related papers  (2025-02-28T14:32:04Z)
• Vision-Based Deep Reinforcement Learning of UAV Autonomous Navigation Using Privileged Information [6.371251946803415]
  DPRL is an end-to-end policy designed to address the challenge of high-speed autonomous UAV navigation under partially observable environmental conditions.<n>We leverage an asymmetric Actor-Critic architecture to provide the agent with privileged information during training.<n>We conduct extensive simulations across various scenarios, benchmarking our DPRL algorithm against the state-of-the-art navigation algorithms.
  arXiv  Detail & Related papers  (2024-12-09T09:05:52Z)
• A Visual-inertial Localization Algorithm using Opportunistic Visual Beacons and Dead-Reckoning for GNSS-Denied Large-scale Applications [1.6599082439305994]
  Augmented reality (AR) allows pedestrians to acquire real-time visual information.<n>We propose a low-cost visual-inertial positioning solution.
  arXiv  Detail & Related papers  (2024-11-29T17:04:03Z)
• Deep Heuristic Learning for Real-Time Urban Pathfinding [0.0]
  This paper introduces a novel approach to urban pathfinding by transforming traditional algorithms into deep learning models that leverage real-time contextual data, such as traffic and weather conditions. We propose two methods: an enhanced A* algorithm that dynamically adjusts routes based on current environmental conditions, and a neural network model that predicts the next optimal path segment using historical and live data.
  arXiv  Detail & Related papers  (2024-11-07T00:22:04Z)
• SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning [49.83621156017321]
  SimBa is an architecture designed to scale up parameters in deep RL by injecting a simplicity bias.<n>By scaling up parameters with SimBa, the sample efficiency of various deep RL algorithms-including off-policy, on-policy, and unsupervised methods-is consistently improved.
  arXiv  Detail & Related papers  (2024-10-13T07:20:53Z)
• Navigation in a simplified Urban Flow through Deep Reinforcement Learning [0.9217021281095907]
  Unmanned aerial vehicles (UAVs) in urban environments require a strategy to minimize their environmental impact. Our goal is to develop DRL algorithms capable of enabling the autonomous navigation of UAVs in urban environments.
  arXiv  Detail & Related papers  (2024-09-26T15:05:15Z)
• OPUS: Occupancy Prediction Using a Sparse Set [64.60854562502523]
  We present a framework to simultaneously predict occupied locations and classes using a set of learnable queries. OPUS incorporates a suite of non-trivial strategies to enhance model performance. Our lightest model achieves superior RayIoU on the Occ3D-nuScenes dataset at near 2x FPS, while our heaviest model surpasses previous best results by 6.1 RayIoU.
  arXiv  Detail & Related papers  (2024-09-14T07:44:22Z)
• Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation [72.24964965882783]
  Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error.<n>Real-world robotic tasks often suffer from sparse rewards, leading to inefficient exploration and suboptimal policies.<n>We introduce Confidence-Controlled Exploration (CCE), a novel method that improves sample efficiency in RL-based robotic navigation without modifying the reward function.
  arXiv  Detail & Related papers  (2023-06-09T18:45:15Z)
• Road Network Guided Fine-Grained Urban Traffic Flow Inference [108.64631590347352]
  Accurate inference of fine-grained traffic flow from coarse-grained one is an emerging yet crucial problem. We propose a novel Road-Aware Traffic Flow Magnifier (RATFM) that exploits the prior knowledge of road networks. Our method can generate high-quality fine-grained traffic flow maps.
  arXiv  Detail & Related papers  (2021-09-29T07:51:49Z)
• Trajectory Design for UAV-Based Internet-of-Things Data Collection: A Deep Reinforcement Learning Approach [93.67588414950656]
  In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a 3D environment. We present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm. Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.
  arXiv  Detail & Related papers  (2021-07-23T03:33:29Z)
• Congestion-aware Evacuation Routing using Augmented Reality Devices [96.68280427555808]
  We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes among multiple destinations. A population density map, obtained on-the-fly by aggregating locations of evacuees from user-end Augmented Reality (AR) devices, is used to model the congestion distribution inside a building.
  arXiv  Detail & Related papers  (2020-04-25T22:54:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.