Deep Learning based Multi-Modal Sensing for Tracking and State Extraction of Small Quadcopters

• URL: http://arxiv.org/abs/2012.04794v1
• Date: Tue, 8 Dec 2020 23:59:48 GMT
• Title: Deep Learning based Multi-Modal Sensing for Tracking and State Extraction of Small Quadcopters
• Authors: Zhibo Zhang, Chen Zeng, Maulikkumar Dhameliya, Souma Chowdhury, Rahul Rai
• Abstract summary: This paper proposes a multi-sensor based approach to detect, track, and localize a quadcopter unmanned aerial vehicle (UAV) Specifically, a pipeline is developed to process monocular RGB and thermal video (captured from a fixed platform) to detect and track the UAV in our FoV. A 2D planar lidar is used to allow conversion of pixel data to actual distance measurements, and thereby enable localization of the UAV in global coordinates.
• Score: 3.019035926889528
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a multi-sensor based approach to detect, track, and localize a quadcopter unmanned aerial vehicle (UAV). Specifically, a pipeline is developed to process monocular RGB and thermal video (captured from a fixed platform) to detect and track the UAV in our FoV. Subsequently, a 2D planar lidar is used to allow conversion of pixel data to actual distance measurements, and thereby enable localization of the UAV in global coordinates. The monocular data is processed through a deep learning-based object detection method that computes an initial bounding box for the UAV. The thermal data is processed through a thresholding and Kalman filter approach to detect and track the bounding box. Training and testing data are prepared by combining a set of original experiments conducted in a motion capture environment and publicly available UAV image data. The new pipeline compares favorably to existing methods and demonstrates promising tracking and localization capacity of sample experiments.

Related papers

• UAVDB: Trajectory-Guided Adaptable Bounding Boxes for UAV Detection [0.03464344220266879]
  Patch Intensity Convergence (PIC) technique generates high-fidelity bounding boxes for UAV detection without manual labeling. This technique forms the foundation of UAVDB, a dedicated database designed specifically for UAV detection. We benchmark UAVDB using state-of-the-art (SOTA) YOLO series detectors, providing a comprehensive performance analysis.
  arXiv  Detail & Related papers  (2024-09-09T13:27:53Z)
• UAV-Based Human Body Detector Selection and Fusion for Geolocated Saliency Map Generation [0.2499907423888049]
  The problem of reliably detecting and geolocating objects of different classes in soft real-time is essential in many application areas, such as Search and Rescue performed using Unmanned Aerial Vehicles (UAVs) This research addresses the complementary problems of system contextual vision-based detector selection, allocation, and execution. The detection results are fused using a method for building maps of salient locations which takes advantage of a novel sensor model for vision-based detections for both positive and negative observations.
  arXiv  Detail & Related papers  (2024-08-29T13:00:37Z)
• Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge [20.459377705070043]
  This report presents the 1st winning model for UG2+, a task in CVPR 2024 UAV Tracking and Pose-Estimation Challenge. We propose a multi-modal UAV detection, classification, and 3D tracking method for accurate UAV classification and tracking. Our system integrates cutting-edge classification techniques and sophisticated post-processing steps to boost accuracy and robustness.
  arXiv  Detail & Related papers  (2024-05-26T07:21:18Z)
• Radio Map Estimation -- An Open Dataset with Directive Transmitter Antennas and Initial Experiments [49.61405888107356]
  We release a dataset of simulated path loss radio maps together with realistic city maps from real-world locations and aerial images from open datasources. Initial experiments regarding model architectures, input feature design and estimation of radio maps from aerial images are presented.
  arXiv  Detail & Related papers  (2024-01-12T14:56:45Z)
• Deep Learning Computer Vision Algorithms for Real-time UAVs On-board Camera Image Processing [77.34726150561087]
  This paper describes how advanced deep learning based computer vision algorithms are applied to enable real-time on-board sensor processing for small UAVs. All algorithms have been developed using state-of-the-art image processing methods based on deep neural networks.
  arXiv  Detail & Related papers  (2022-11-02T11:10:42Z)
• Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images [96.66271207089096]
  FCOS-LiDAR is a fully convolutional one-stage 3D object detector for LiDAR point clouds of autonomous driving scenes. We show that an RV-based 3D detector with standard 2D convolutions alone can achieve comparable performance to state-of-the-art BEV-based detectors.
  arXiv  Detail & Related papers  (2022-05-27T05:42:16Z)
• A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data [0.17205106391379021]
  This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene.
  arXiv  Detail & Related papers  (2021-08-27T16:16:01Z)
• Brain over Brawn -- Using a Stereo Camera to Detect, Track and Intercept a Faster UAV by Reconstructing Its Trajectory [2.1638817206926855]
  This paper demonstrates our approach to intercepting a faster intruder UAV, inspired by the MBZIRC 2020 Challenge 1. By leveraging the shape of the intruder's trajectory we are able to calculate the interception point. The system is able to track and intercept the target which is 30% faster than the interceptor in majority of simulation experiments.
  arXiv  Detail & Related papers  (2021-07-02T10:49:22Z)
• Self-Supervised Person Detection in 2D Range Data using a Calibrated Camera [83.31666463259849]
  We propose a method to automatically generate training labels (called pseudo-labels) for 2D LiDAR-based person detectors. We show that self-supervised detectors, trained or fine-tuned with pseudo-labels, outperform detectors trained using manual annotations. Our method is an effective way to improve person detectors during deployment without any additional labeling effort.
  arXiv  Detail & Related papers  (2020-12-16T12:10:04Z)
• Perceiving Traffic from Aerial Images [86.994032967469]
  We propose an object detection method called Butterfly Detector that is tailored to detect objects in aerial images. We evaluate our Butterfly Detector on two publicly available UAV datasets (UAVDT and VisDrone 2019) and show that it outperforms previous state-of-the-art methods while remaining real-time.
  arXiv  Detail & Related papers  (2020-09-16T11:37:43Z)
• DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes [54.239416488865565]
  We propose a fast single-stage 3D object detection method for LIDAR data. The core novelty of our method is a fast, single-pass architecture that both detects objects in 3D and estimates their shapes. We find that our proposed method achieves state-of-the-art results by 5% on object detection in ScanNet scenes, and it gets top results by 3.4% in the Open dataset.
  arXiv  Detail & Related papers  (2020-04-02T17:48:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.