A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles

• URL: http://arxiv.org/abs/2503.05251v1
• Date: Fri, 07 Mar 2025 09:07:07 GMT
• Title: A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles
• Authors: Lorenzo Scarciglia, Antonio Paolillo, Daniele Palossi,
• Abstract summary: Palm-sized autonomous nano-drones, i.e., sub-50g in weight, recently entered the drone racing scenario, where they are tasked to avoid obstacles and navigate as fast as possible through gates.<n>This work presents a map-free vision-based autonomous nano-drone that combines a real-time deep learning gate detection front-end with a classic yet elegant and effective visual servoing control back-end.<n>In-field experiments highlight the capability of our nano-drone to successfully navigate through 15 gates in 4 min, never crashing and covering a total travel distance of 100m, with
• Score: 6.894810724560054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Palm-sized autonomous nano-drones, i.e., sub-50g in weight, recently entered the drone racing scenario, where they are tasked to avoid obstacles and navigate as fast as possible through gates. However, in contrast with their bigger counterparts, i.e., kg-scale drones, nano-drones expose three orders of magnitude less onboard memory and compute power, demanding more efficient and lightweight vision-based pipelines to win the race. This work presents a map-free vision-based (using only a monocular camera) autonomous nano-drone that combines a real-time deep learning gate detection front-end with a classic yet elegant and effective visual servoing control back-end, only relying on onboard resources. Starting from two state-of-the-art tiny deep learning models, we adapt them for our specific task, and after a mixed simulator-real-world training, we integrate and deploy them aboard our nano-drone. Our best-performing pipeline costs of only 24M multiply-accumulate operations per frame, resulting in a closed-loop control performance of 30 Hz, while achieving a gate detection root mean square error of 1.4 pixels, on our ~20k real-world image dataset. In-field experiments highlight the capability of our nano-drone to successfully navigate through 15 gates in 4 min, never crashing and covering a total travel distance of ~100m, with a peak flight speed of 1.9 m/s. Finally, to stress the generalization capability of our system, we also test it in a never-seen-before environment, where it navigates through gates for more than 4 min.

Related papers

• Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW [52.280742520586756]
  Miniaturized cyber-physical systems (CPSes) powered by tiny machine learning (TinyML), such as nano-drones, are becoming an increasingly attractive technology. Simple electronics make these CPSes inexpensive, but strongly limit the computational, memory, and sensing resources available on board. We present a novel on-device fine-tuning approach that relies only on the limited ultra-low power resources available aboard nano-drones.
  arXiv  Detail & Related papers  (2024-08-06T13:11:36Z)
• High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks [51.23613834703353]
  Relative drone-to-drone localization is a fundamental building block for any swarm operations. We present a vertically integrated system based on a novel vision-based fully convolutional neural network (FCNN) Our model results in an R-squared improvement from 32 to 47% on the horizontal image coordinate and from 18 to 55% on the vertical image coordinate, on a real-world dataset of 30k images.
  arXiv  Detail & Related papers  (2024-02-21T12:34:31Z)
• A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones [12.439787085435661]
  We propose A3D, an edge server assisted drone navigation framework. A3D can reduce end-to-end latency by 28.06% and extend the flight distance by up to 27.28% compared with non-adaptive solutions.
  arXiv  Detail & Related papers  (2023-07-19T10:23:28Z)
• Ultra-low Power Deep Learning-based Monocular Relative Localization Onboard Nano-quadrotors [64.68349896377629]
  This work presents a novel autonomous end-to-end system that addresses the monocular relative localization, through deep neural networks (DNNs), of two peer nano-drones. To cope with the ultra-constrained nano-drone platform, we propose a vertically-integrated framework, including dataset augmentation, quantization, and system optimizations. Experimental results show that our DNN can precisely localize a 10cm-size target nano-drone by employing only low-resolution monochrome images, up to 2m distance.
  arXiv  Detail & Related papers  (2023-03-03T14:14:08Z)
• TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos [57.92385818430939]
  Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
  arXiv  Detail & Related papers  (2022-10-16T03:05:13Z)
• Coupling Vision and Proprioception for Navigation of Legged Robots [65.59559699815512]
  We exploit the complementary strengths of vision and proprioception to achieve point goal navigation in a legged robot. We show superior performance compared to wheeled robot (LoCoBot) baselines. We also show the real-world deployment of our system on a quadruped robot with onboard sensors and compute.
  arXiv  Detail & Related papers  (2021-12-03T18:59:59Z)
• AlphaPilot: Autonomous Drone Racing [47.205375478625776]
  The system has successfully been deployed at the first autonomous drone racing world championship: the 2019 AlphaPilot Challenge. The proposed system has been demonstrated to successfully guide the drone through tight race courses reaching speeds up to 8m/s.
  arXiv  Detail & Related papers  (2020-05-26T15:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.