Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW

• URL: http://arxiv.org/abs/2408.03168v1
• Date: Tue, 6 Aug 2024 13:11:36 GMT
• Title: Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW
• Authors: Elia Cereda, Alessandro Giusti, Daniele Palossi,
• Abstract summary: Miniaturized cyber-physical systems (CPSes) powered by tiny machine learning (TinyML), such as nano-drones, are becoming an increasingly attractive technology. Simple electronics make these CPSes inexpensive, but strongly limit the computational, memory, and sensing resources available on board. We present a novel on-device fine-tuning approach that relies only on the limited ultra-low power resources available aboard nano-drones.
• Score: 52.280742520586756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Miniaturized cyber-physical systems (CPSes) powered by tiny machine learning (TinyML), such as nano-drones, are becoming an increasingly attractive technology. Their small form factor (i.e., ~10cm diameter) ensures vast applicability, ranging from the exploration of narrow disaster scenarios to safe human-robot interaction. Simple electronics make these CPSes inexpensive, but strongly limit the computational, memory, and sensing resources available on board. In real-world applications, these limitations are further exacerbated by domain shift. This fundamental machine learning problem implies that model perception performance drops when moving from the training domain to a different deployment one. To cope with and mitigate this general problem, we present a novel on-device fine-tuning approach that relies only on the limited ultra-low power resources available aboard nano-drones. Then, to overcome the lack of ground-truth training labels aboard our CPS, we also employ a self-supervised method based on ego-motion consistency. Albeit our work builds on top of a specific real-world vision-based human pose estimation task, it is widely applicable for many embedded TinyML use cases. Our 512-image on-device training procedure is fully deployed aboard an ultra-low power GWT GAP9 System-on-Chip and requires only 1MB of memory while consuming as low as 19mW or running in just 510ms (at 38mW). Finally, we demonstrate the benefits of our on-device learning approach by field-testing our closed-loop CPS, showing a reduction in horizontal position error of up to 26% vs. a non-fine-tuned state-of-the-art baseline. In the most challenging never-seen-before environment, our on-device learning procedure makes the difference between succeeding or failing the mission.

Related papers

• Tiny-PULP-Dronets: Squeezing Neural Networks for Faster and Lighter Inference on Multi-Tasking Autonomous Nano-Drones [12.96119439129453]
  This work moves from PULP-Dronet, a State-of-the-Art convolutional neural network for autonomous navigation on nano-drones, to Tiny-PULP-Dronet, a novel methodology to squeeze by more than one order of magnitude model size. This massive reduction paves the way towards affordable multi-tasking on nano-drones, a fundamental requirement for achieving high-level intelligence.
  arXiv  Detail & Related papers  (2024-07-02T16:24:57Z)
• On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors [53.59319391812798]
  Sub-SI50gram nano-drones are gaining momentum in both academia and industry. Their most compelling applications rely on onboard deep learning models for perception. When deployed in unknown environments, these models often underperform due to domain shift. We propose for the first time, on-device learning aboard nano-drones, where the first part of the in-field mission is dedicated to self-supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-03-06T22:04:14Z)
• High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks [51.23613834703353]
  Relative drone-to-drone localization is a fundamental building block for any swarm operations. We present a vertically integrated system based on a novel vision-based fully convolutional neural network (FCNN) Our model results in an R-squared improvement from 32 to 47% on the horizontal image coordinate and from 18 to 55% on the vertical image coordinate, on a real-world dataset of 30k images.
  arXiv  Detail & Related papers  (2024-02-21T12:34:31Z)
• Adaptive Deep Learning for Efficient Visual Pose Estimation aboard Ultra-low-power Nano-drones [5.382126081742012]
  We present a novel adaptive deep learning-based mechanism for the efficient execution of a vision-based human pose estimation task. On a real-world dataset and the actual nano-drone hardware, our best-performing system shows 28% latency reduction while keeping the same mean absolute error (MAE), 3% MAE reduction while being iso-latency, and the absolute peak performance, i.e., 6% better than SoA model.
  arXiv  Detail & Related papers  (2024-01-26T23:04:26Z)
• Ultra-low Power Deep Learning-based Monocular Relative Localization Onboard Nano-quadrotors [64.68349896377629]
  This work presents a novel autonomous end-to-end system that addresses the monocular relative localization, through deep neural networks (DNNs), of two peer nano-drones. To cope with the ultra-constrained nano-drone platform, we propose a vertically-integrated framework, including dataset augmentation, quantization, and system optimizations. Experimental results show that our DNN can precisely localize a 10cm-size target nano-drone by employing only low-resolution monochrome images, up to 2m distance.
  arXiv  Detail & Related papers  (2023-03-03T14:14:08Z)
• Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs [69.19616451596342]
  Miniaturized unmanned aerial vehicles (UAVs) are an emerging and trending topic. We leverage a novel neural architecture search (NAS) technique to automatically identify several convolutional neural networks (CNNs) for a visual pose estimation task. Our results improve the State-of-the-Art by reducing the in-field control error of 32% while achieving a real-time onboard inference-rate of 10Hz@10mW and 50Hz@90mW.
  arXiv  Detail & Related papers  (2023-03-03T14:02:09Z)
• Robustifying the Deployment of tinyML Models for Autonomous mini-vehicles [61.27933385742613]
  We propose a closed-loop learning flow for autonomous driving mini-vehicles that includes the target environment in-the-loop. We leverage a family of tinyCNNs to control the mini-vehicle, which learn in the target environment by imitating a computer vision algorithm, i.e., the expert. When running the family of CNNs, our solution outperforms any other implementation on the STM32L4 and k64f (Cortex-M4), reducing the latency by over 13x and the energy consummation by 92%.
  arXiv  Detail & Related papers  (2020-07-01T07:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.