Vision-based Drone Flocking in Outdoor Environments
- URL: http://arxiv.org/abs/2012.01245v2
- Date: Tue, 16 Feb 2021 10:13:38 GMT
- Title: Vision-based Drone Flocking in Outdoor Environments
- Authors: Fabian Schilling, Fabrizio Schiano, Dario Floreano
- Abstract summary: This letter proposes a vision-based detection and tracking algorithm for drone swarms.
We employ a convolutional neural network to detect and localize nearby agents onboard the quadcopters in real-time.
We show that the drones can safely navigate in an outdoor environment despite substantial background clutter and difficult lighting conditions.
- Score: 9.184987303791292
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Decentralized deployment of drone swarms usually relies on inter-agent
communication or visual markers that are mounted on the vehicles to simplify
their mutual detection. This letter proposes a vision-based detection and
tracking algorithm that enables groups of drones to navigate without
communication or visual markers. We employ a convolutional neural network to
detect and localize nearby agents onboard the quadcopters in real-time. Rather
than manually labeling a dataset, we automatically annotate images to train the
neural network using background subtraction by systematically flying a
quadcopter in front of a static camera. We use a multi-agent state tracker to
estimate the relative positions and velocities of nearby agents, which are
subsequently fed to a flocking algorithm for high-level control. The drones are
equipped with multiple cameras to provide omnidirectional visual inputs. The
camera setup ensures the safety of the flock by avoiding blind spots regardless
of the agent configuration. We evaluate the approach with a group of three real
quadcopters that are controlled using the proposed vision-based flocking
algorithm. The results show that the drones can safely navigate in an outdoor
environment despite substantial background clutter and difficult lighting
conditions. The source code, image dataset, and trained detection model are
available at https://github.com/lis-epfl/vswarm.
Related papers
- PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot [3.387892563308912]
We introduce a novel approach to process a sequence of dynamic successive frames in a line-of-sight (LOS) video using an attention-based neural network.
We validate the approach on in-the-wild scenes using a drone for video capture, thus demonstrating low-cost NLOS imaging in dynamic capture environments.
arXiv Detail & Related papers (2024-04-07T17:31:53Z) - Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone
Racing [52.50284630866713]
Existing systems often require hand-engineered components for state estimation, planning, and control.
This paper tackles the vision-based autonomous-drone-racing problem by learning deep sensorimotor policies.
arXiv Detail & Related papers (2022-10-26T19:03:17Z) - TransVisDrone: Spatio-Temporal Transformer for Vision-based
Drone-to-Drone Detection in Aerial Videos [57.92385818430939]
Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones.
Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices.
We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
arXiv Detail & Related papers (2022-10-16T03:05:13Z) - Drone Detection and Tracking in Real-Time by Fusion of Different Sensing
Modalities [66.4525391417921]
We design and evaluate a multi-sensor drone detection system.
Our solution integrates a fish-eye camera as well to monitor a wider part of the sky and steer the other cameras towards objects of interest.
The thermal camera is shown to be a feasible solution as good as the video camera, even if the camera employed here has a lower resolution.
arXiv Detail & Related papers (2022-07-05T10:00:58Z) - Dogfight: Detecting Drones from Drones Videos [58.158988162743825]
This paper attempts to address the problem of drones detection from other flying drones variations.
The erratic movement of the source and target drones, small size, arbitrary shape, large intensity, and occlusion make this problem quite challenging.
To handle this, instead of using region-proposal based methods, we propose to use a two-stage segmentation-based approach.
arXiv Detail & Related papers (2021-03-31T17:43:31Z) - Learn by Observation: Imitation Learning for Drone Patrolling from
Videos of A Human Navigator [22.06785798356346]
We propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground.
The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency.
A newly designed neural network is trained based on the annotated data to predict appropriate directions and translations.
arXiv Detail & Related papers (2020-08-30T15:20:40Z) - Real-Time Drone Detection and Tracking With Visible, Thermal and
Acoustic Sensors [66.4525391417921]
A thermal infrared camera is shown to be a feasible solution to the drone detection task.
The detector performance as a function of the sensor-to-target distance is also investigated.
A novel video dataset containing 650 annotated infrared and visible videos of drones, birds, airplanes and helicopters is also presented.
arXiv Detail & Related papers (2020-07-14T23:06:42Z) - Built Infrastructure Monitoring and Inspection Using UAVs and
Vision-based Algorithms [2.0305676256390934]
This study presents an inspecting system using real-time control unmanned aerial vehicles (UAVs) to investigate structural surfaces.
The system operates under favourable weather conditions to inspect a target structure, which is the Wentworth light rail base structure in this study.
arXiv Detail & Related papers (2020-05-19T14:37:48Z) - Detection and Tracking Meet Drones Challenge [131.31749447313197]
This paper presents a review of object detection and tracking datasets and benchmarks, and discusses the challenges of collecting large-scale drone-based object detection and tracking datasets with manual annotations.
We describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South.
We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions.
arXiv Detail & Related papers (2020-01-16T00:11:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.