Deep Learning on Home Drone: Searching for the Optimal Architecture
- URL: http://arxiv.org/abs/2209.11064v1
- Date: Wed, 21 Sep 2022 11:41:45 GMT
- Title: Deep Learning on Home Drone: Searching for the Optimal Architecture
- Authors: Alaa Maalouf and Yotam Gurfinkel and Barak Diker and Oren Gal and
Daniela Rus and Dan Feldman
- Abstract summary: We suggest the first system that runs real-time semantic segmentation via deep learning on a weak micro-computer such as the Raspberry Pi Zero v2 attached to a toy-drone.
In particular, since the Raspberry Pi weighs less than $16$ grams, and its size is half of a credit card, we could easily attach it to the common commercial DJI Tello toy-drone.
The result is an autonomous drone that can detect and classify objects in real-time from a video stream of an on-board monocular RGB camera.
- Score: 54.535788447839884
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We suggest the first system that runs real-time semantic segmentation via
deep learning on a weak micro-computer such as the Raspberry Pi Zero v2 (whose
price was \$15) attached to a toy-drone. In particular, since the Raspberry Pi
weighs less than $16$ grams, and its size is half of a credit card, we could
easily attach it to the common commercial DJI Tello toy-drone (<\$100, <90
grams, 98 $\times$ 92.5 $\times$ 41 mm). The result is an autonomous drone (no
laptop nor human in the loop) that can detect and classify objects in real-time
from a video stream of an on-board monocular RGB camera (no GPS or LIDAR
sensors). The companion videos demonstrate how this Tello drone scans the lab
for people (e.g. for the use of firefighters or security forces) and for an
empty parking slot outside the lab.
Existing deep learning solutions are either much too slow for real-time
computation on such IoT devices, or provide results of impractical quality. Our
main challenge was to design a system that takes the best of all worlds among
numerous combinations of networks, deep learning platforms/frameworks,
compression techniques, and compression ratios. To this end, we provide an
efficient searching algorithm that aims to find the optimal combination which
results in the best tradeoff between the network running time and its
accuracy/performance.
Related papers
- High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks [51.23613834703353]
Relative drone-to-drone localization is a fundamental building block for any swarm operations.
We present a vertically integrated system based on a novel vision-based fully convolutional neural network (FCNN)
Our model results in an R-squared improvement from 32 to 47% on the horizontal image coordinate and from 18 to 55% on the vertical image coordinate, on a real-world dataset of 30k images.
arXiv Detail & Related papers (2024-02-21T12:34:31Z) - Channel-Aware Distillation Transformer for Depth Estimation on Nano
Drones [9.967643080731683]
This paper presents a lightweight CNN depth estimation network deployed on nano drones for obstacle avoidance.
Inspired by Knowledge Distillation (KD), a Channel-Aware Distillation Transformer (CADiT) is proposed to facilitate the small network.
The proposed method is validated on the KITTI dataset and tested on a nano drone Crazyflie, with an ultra-low power microprocessor GAP8.
arXiv Detail & Related papers (2023-03-18T10:45:34Z) - TransVisDrone: Spatio-Temporal Transformer for Vision-based
Drone-to-Drone Detection in Aerial Videos [57.92385818430939]
Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones.
Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices.
We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
arXiv Detail & Related papers (2022-10-16T03:05:13Z) - Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge
TPU [58.720142291102135]
In this paper we propose a pose estimation software exploiting neural network architectures.
We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
arXiv Detail & Related papers (2022-04-07T08:53:18Z) - Newton-PnP: Real-time Visual Navigation for Autonomous Toy-Drones [15.075691719756877]
Perspective-n-Point problem aims to estimate the relative pose between a calibrated monocular camera and a known 3D model.
We suggest an algorithm that runs on weak IoT in real-time but still provides provable guarantees for both running time and correctness.
Our main motivation was to turn the popular DJI's Tello Drone into an autonomous drone that navigates in an indoor environment with no external human/laptop/sensor.
arXiv Detail & Related papers (2022-03-05T09:00:50Z) - A Real-time Low-cost Artificial Intelligence System for Autonomous
Spraying in Palm Plantations [1.6799377888527687]
In precision crop protection, (target-orientated) object detection in image processing can help navigate Unmanned Aerial Vehicles (UAV, crop protection drones) to the right place to apply the pesticide.
We propose a solution based on a light deep neural network (DNN), called Ag-YOLO, which can make the crop protection UAV have the ability to target detection and autonomous operation.
arXiv Detail & Related papers (2021-03-06T15:05:14Z) - Multi-Drone based Single Object Tracking with Agent Sharing Network [74.8198920355117]
Multi-Drone single Object Tracking dataset consists of 92 groups of video clips with 113,918 high resolution frames taken by two drones and 63 groups of video clips with 145,875 high resolution frames taken by three drones.
Agent sharing network (ASNet) is proposed by self-supervised template sharing and view-aware fusion of the target from multiple drones.
arXiv Detail & Related papers (2020-03-16T03:27:04Z) - University-1652: A Multi-view Multi-source Benchmark for Drone-based
Geo-localization [87.74121935246937]
We introduce a new multi-view benchmark for drone-based geo-localization, named University-1652.
University-1652 contains data from three platforms, i.e., synthetic drones, satellites and ground cameras of 1,652 university buildings around the world.
Experiments show that University-1652 helps the model to learn the viewpoint-invariant features and also has good generalization ability in the real-world scenario.
arXiv Detail & Related papers (2020-02-27T15:24:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.