TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos

• URL: http://arxiv.org/abs/2210.08423v2
• Date: Sat, 26 Aug 2023 00:54:05 GMT
• Title: TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos
• Authors: Tushar Sangam, Ishan Rajendrakumar Dave, Waqas Sultani, Mubarak Shah
• Abstract summary: Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
• Score: 57.92385818430939
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. However, existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. In this work, we propose a simple yet effective framework, \textit{TransVisDrone}, that provides an end-to-end solution with higher computational efficiency. We utilize CSPDarkNet-53 network to learn object-related spatial features and VideoSwin model to improve drone detection in challenging scenarios by learning spatio-temporal dependencies of drone motion. Our method achieves state-of-the-art performance on three challenging real-world datasets (Average Precision@0.5IOU): NPS 0.95, FLDrones 0.75, and AOT 0.80, and a higher throughput than previous methods. We also demonstrate its deployment capability on edge devices and its usefulness in detecting drone-collision (encounter). Project: \url{https://tusharsangam.github.io/TransVisDrone-project-page/}.

Related papers

• C2FDrone: Coarse-to-Fine Drone-to-Drone Detection using Vision Transformer Networks [23.133250476580038]
  A vision-based drone-to-drone detection system is crucial for various applications like collision avoidance, countering hostile drones, and search-and-rescue operations. detecting drones presents unique challenges, including small object sizes, distortion, and real-time processing requirements. We propose a novel coarse-to-fine detection strategy based on vision transformers.
  arXiv  Detail & Related papers  (2024-04-30T05:51:21Z)
• High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks [51.23613834703353]
  Relative drone-to-drone localization is a fundamental building block for any swarm operations. We present a vertically integrated system based on a novel vision-based fully convolutional neural network (FCNN) Our model results in an R-squared improvement from 32 to 47% on the horizontal image coordinate and from 18 to 55% on the vertical image coordinate, on a real-world dataset of 30k images.
  arXiv  Detail & Related papers  (2024-02-21T12:34:31Z)
• A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones [12.439787085435661]
  We propose A3D, an edge server assisted drone navigation framework. A3D can reduce end-to-end latency by 28.06% and extend the flight distance by up to 27.28% compared with non-adaptive solutions.
  arXiv  Detail & Related papers  (2023-07-19T10:23:28Z)
• AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning [63.628195002143734]
  We propose a novel approach for aerial video action recognition. Our method is designed for videos captured using UAVs and can run on edge or mobile devices. We present a learning-based approach that uses customized auto zoom to automatically identify the human target and scale it appropriately.
  arXiv  Detail & Related papers  (2023-03-02T21:24:19Z)
• Lightweight Multi-Drone Detection and 3D-Localization via YOLO [1.284647943889634]
  We present and evaluate a method to perform real-time multiple drone detection and three-dimensional localization. We use state-of-the-art tiny-YOLOv4 object detection algorithm and stereo triangulation. Our computer vision approach eliminates the need for computationally expensive stereo matching algorithms.
  arXiv  Detail & Related papers  (2022-02-18T09:41:23Z)
• Track Boosting and Synthetic Data Aided Drone Detection [0.0]
  Our method approaches the drone detection problem by fine-tuning a YOLOv5 model with real and synthetically generated data. Our results indicate that augmenting the real data with an optimal subset of synthetic data can increase the performance.
  arXiv  Detail & Related papers  (2021-11-24T10:16:27Z)
• Dogfight: Detecting Drones from Drones Videos [58.158988162743825]
  This paper attempts to address the problem of drones detection from other flying drones variations. The erratic movement of the source and target drones, small size, arbitrary shape, large intensity, and occlusion make this problem quite challenging. To handle this, instead of using region-proposal based methods, we propose to use a two-stage segmentation-based approach.
  arXiv  Detail & Related papers  (2021-03-31T17:43:31Z)
• Multi-Drone based Single Object Tracking with Agent Sharing Network [74.8198920355117]
  Multi-Drone single Object Tracking dataset consists of 92 groups of video clips with 113,918 high resolution frames taken by two drones and 63 groups of video clips with 145,875 high resolution frames taken by three drones. Agent sharing network (ASNet) is proposed by self-supervised template sharing and view-aware fusion of the target from multiple drones.
  arXiv  Detail & Related papers  (2020-03-16T03:27:04Z)
• Detection and Tracking Meet Drones Challenge [131.31749447313197]
  This paper presents a review of object detection and tracking datasets and benchmarks, and discusses the challenges of collecting large-scale drone-based object detection and tracking datasets with manual annotations. We describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions.
  arXiv  Detail & Related papers  (2020-01-16T00:11:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.