Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge

• URL: http://arxiv.org/abs/2405.16464v1
• Date: Sun, 26 May 2024 07:21:18 GMT
• Title: Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge
• Authors: Tianchen Deng, Yi Zhou, Wenhua Wu, Mingrui Li, Jingwei Huang, Shuhong Liu, Yanzeng Song, Hao Zuo, Yanbo Wang, Yutao Yue, Hesheng Wang, Weidong Chen,
• Abstract summary: This report presents the 1st winning model for UG2+, a task in CVPR 2024 UAV Tracking and Pose-Estimation Challenge. We propose a multi-modal UAV detection, classification, and 3D tracking method for accurate UAV classification and tracking. Our system integrates cutting-edge classification techniques and sophisticated post-processing steps to boost accuracy and robustness.
• Score: 20.459377705070043
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This technical report presents the 1st winning model for UG2+, a task in CVPR 2024 UAV Tracking and Pose-Estimation Challenge. This challenge faces difficulties in drone detection, UAV-type classification and 2D/3D trajectory estimation in extreme weather conditions with multi-modal sensor information, including stereo vision, various Lidars, Radars, and audio arrays. Leveraging this information, we propose a multi-modal UAV detection, classification, and 3D tracking method for accurate UAV classification and tracking. A novel classification pipeline which incorporates sequence fusion, region of interest (ROI) cropping, and keyframe selection is proposed. Our system integrates cutting-edge classification techniques and sophisticated post-processing steps to boost accuracy and robustness. The designed pose estimation pipeline incorporates three modules: dynamic points analysis, a multi-object tracker, and trajectory completion techniques. Extensive experiments have validated the effectiveness and precision of our approach. In addition, we also propose a novel dataset pre-processing method and conduct a comprehensive ablation study for our design. We finally achieved the best performance in the classification and tracking of the MMUAD dataset. The code and configuration of our method are available at https://github.com/dtc111111/Multi-Modal-UAV.

Related papers

• Clustering-based Learning for UAV Tracking and Pose Estimation [0.0]
  This work develops a clustering-based learning detection approach, CL-Det, for UAV tracking and pose estimation using two types of LiDARs. We first align the timestamps of Livox Avia data and LiDAR 360 data and then separate the point cloud of objects of interest (OOIs) from the environment. The proposed method shows competitive pose estimation performance and ranks 5th on the final leaderboard of the CVPR 2024 UG2+ Challenge.
  arXiv  Detail & Related papers  (2024-05-27T06:33:25Z)
• Lifting Multi-View Detection and Tracking to the Bird's Eye View [5.679775668038154]
  Recent advancements in multi-view detection and 3D object recognition have significantly improved performance. We compare modern lifting methods, both parameter-free and parameterized, to multi-view aggregation. We present an architecture that aggregates the features of multiple times steps to learn robust detection.
  arXiv  Detail & Related papers  (2024-03-19T09:33:07Z)
• UniMODE: Unified Monocular 3D Object Detection [70.27631528933482]
  We build a detector based on the bird's-eye-view (BEV) detection paradigm. We propose an uneven BEV grid design to handle the convergence instability caused by the challenges. A unified detector UniMODE is derived, which surpasses the previous state-of-the-art on the challenging Omni3D dataset.
  arXiv  Detail & Related papers  (2024-02-28T18:59:31Z)
• MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threats [37.981623262267036]
  MMAUD addresses a critical gap in contemporary threat detection methodologies by focusing on drone detection, UAV-type classification, and trajectory estimation. It offers a unique overhead aerial detection vital for addressing real-world scenarios with higher fidelity than datasets captured on specific vantage points using thermal and RGB. Our proposed modalities are cost-effective and highly adaptable, allowing users to experiment and implement new UAV threat detection tools.
  arXiv  Detail & Related papers  (2024-02-06T04:57:07Z)
• You Only Need Two Detectors to Achieve Multi-Modal 3D Multi-Object Tracking [9.20064374262956]
  The proposed framework can achieve robust tracking by using only a 2D detector and a 3D detector. It is proven more accurate than many of the state-of-the-art TBD-based multi-modal tracking methods.
  arXiv  Detail & Related papers  (2023-04-18T02:45:18Z)
• 3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds [95.54285993019843]
  We propose a method for joint detection and tracking of multiple objects in 3D point clouds. Our model exploits temporal information employing multiple frames to detect objects and track them in a single network.
  arXiv  Detail & Related papers  (2022-11-01T20:59:38Z)
• Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline [80.13652104204691]
  In this paper, we construct a large-scale benchmark with high diversity for visible-thermal UAV tracking (VTUAV) We provide a coarse-to-fine attribute annotation, where frame-level attributes are provided to exploit the potential of challenge-specific trackers. In addition, we design a new RGB-T baseline, named Hierarchical Multi-modal Fusion Tracker (HMFT), which fuses RGB-T data in various levels.
  arXiv  Detail & Related papers  (2022-04-08T15:22:33Z)
• VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention [32.44687996180621]
  We propose to adaptively fuse multi-view features in a global spatial context via Dual Cross-VIew SpaTial Attention (VISTA) The proposed VISTA is a novel plug-and-play fusion module, wherein the multi-layer perceptron widely adopted in standard attention modules is replaced with a convolutional one. At the time of submission, our method achieves 63.0% in overall mAP and 69.8% in NDS on the nuScenes benchmark, outperforming all published methods by up to 24% in safety-crucial categories such as cyclist.
  arXiv  Detail & Related papers  (2022-03-18T02:34:59Z)
• Trajectory Design for UAV-Based Internet-of-Things Data Collection: A Deep Reinforcement Learning Approach [93.67588414950656]
  In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a 3D environment. We present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm. Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.
  arXiv  Detail & Related papers  (2021-07-23T03:33:29Z)
• Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking [59.06167734555191]
  Unmanned Aerial Vehicle (UAV) offers lots of applications in both commerce and recreation. We consider the task of tracking UAVs, providing rich information such as location and trajectory. We propose a dataset, Anti-UAV, with more than 300 video pairs containing over 580k manually annotated bounding boxes.
  arXiv  Detail & Related papers  (2021-01-21T07:00:15Z)
• Dense Scene Multiple Object Tracking with Box-Plane Matching [73.54369833671772]
  Multiple Object Tracking (MOT) is an important task in computer vision. We propose the Box-Plane Matching (BPM) method to improve the MOT performacne in dense scenes. With the effectiveness of the three modules, our team achieves the 1st place on the Track-1 leaderboard in the ACM MM Grand Challenge HiEve 2020.
  arXiv  Detail & Related papers  (2020-07-30T16:39:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.