Light-Weight Cross-Modal Enhancement Method with Benchmark Construction for UAV-based Open-Vocabulary Object Detection

• URL: http://arxiv.org/abs/2509.06011v2
• Date: Tue, 09 Sep 2025 12:22:18 GMT
• Title: Light-Weight Cross-Modal Enhancement Method with Benchmark Construction for UAV-based Open-Vocabulary Object Detection
• Authors: Zhenhai Weng, Xinjie Li, Can Wu, Weijie He, Jianfeng Lv, Dong Zhou, Zhongliang Yu,
• Abstract summary: We propose a complete UAV-oriented solution that combines both dataset construction and model innovation.<n>First, we design a refined UAV-Label Engine, which efficiently resolves annotation redundancy, inconsistency, and ambiguity.<n>Second, we introduce the Cross-Attention Gated Enhancement (CAGE) module, a lightweight dual-path fusion design that integrates cross-attention, adaptive gating, and global FiLM modulation for robust textvision alignment.
• Score: 6.443926939309045
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-Vocabulary Object Detection (OVD) faces severe performance degradation when applied to UAV imagery due to the domain gap from ground-level datasets. To address this challenge, we propose a complete UAV-oriented solution that combines both dataset construction and model innovation. First, we design a refined UAV-Label Engine, which efficiently resolves annotation redundancy, inconsistency, and ambiguity, enabling the generation of largescale UAV datasets. Based on this engine, we construct two new benchmarks: UAVDE-2M, with over 2.4M instances across 1,800+ categories, and UAVCAP-15K, providing rich image-text pairs for vision-language pretraining. Second, we introduce the Cross-Attention Gated Enhancement (CAGE) module, a lightweight dual-path fusion design that integrates cross-attention, adaptive gating, and global FiLM modulation for robust textvision alignment. By embedding CAGE into the YOLO-World-v2 framework, our method achieves significant gains in both accuracy and efficiency, notably improving zero-shot detection on VisDrone by +5.3 mAP while reducing parameters and GFLOPs, and demonstrating strong cross-domain generalization on SIMD. Extensive experiments and real-world UAV deployment confirm the effectiveness and practicality of our proposed solution for UAV-based OVD

Related papers

• DRL-Enabled Trajectory Planing for UAV-Assisted VLC: Optimal Altitude and Reward Design [35.154994099093244]
  Integration of aerial vehicle (UAV) and visible light communication (VLC) technologies has emerged as a promising solution to offer efficient lighting.<n>This letter investigates the three-dimensional trajectory planning in a UAV-assisted VLC system.
  arXiv  Detail & Related papers  (2026-01-30T03:44:14Z)
• How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline [74.4054700050366]
  Unmanned Aerial Vehicles (UAVs) offer wide-ranging applications but also pose significant safety and privacy violation risks.<n>Current Anti-UAV research primarily focuses on RGB, infrared (IR), or RGB-IR videos captured by fixed ground cameras.<n>We propose a new multi-modal visual tracking task termed UAV-Anti-UAV, which involves a pursuer UAV tracking a target adversarial UAV in the video stream.
  arXiv  Detail & Related papers  (2025-12-08T10:19:54Z)
• A Tri-Modal Dataset and a Baseline System for Tracking Unmanned Aerial Vehicles [74.8162337823142]
  MM-UAV is the first large-scale benchmark for Multi-Modal UAV Tracking.<n>The dataset spans over 30 challenging scenarios, with 1,321 synchronised multi-modal sequences, and more than 2.8 million annotated frames.<n>Accompanying the dataset, we provide a novel multi-modal multi-UAV tracking framework.
  arXiv  Detail & Related papers  (2025-11-23T08:42:17Z)
• More Clear, More Flexible, More Precise: A Comprehensive Oriented Object Detection benchmark for UAV [58.89234732689013]
  CODrone is a comprehensive oriented object detection dataset for UAVs that accurately reflects real-world conditions.<n>It also serves as a new benchmark designed to align with downstream task requirements.<n>We conduct a series of experiments based on 22 classical or SOTA methods to rigorously evaluate CODrone.
  arXiv  Detail & Related papers  (2025-04-28T17:56:02Z)
• MASF-YOLO: An Improved YOLOv11 Network for Small Object Detection on Drone View [0.0]
  We propose a novel object detection network Multi-scale Context Aggregation and Scale-adaptive Fusion YOLO (MASF-YOLO)<n>To tackle the difficulty of detecting small objects in UAV images, we design a Multi-scale Feature Aggregation Module (MFAM), which significantly improves the detection accuracy of small objects.<n>Thirdly, we introduce a Dimension-Aware Selective Integration Module (DASI), which further enhances multi-scale feature fusion capabilities.
  arXiv  Detail & Related papers  (2025-04-25T07:43:33Z)
• DiRecNetV2: A Transformer-Enhanced Network for Aerial Disaster Recognition [4.678150356894011]
  integration of Unmanned Aerial Vehicles with artificial intelligence (AI) models for aerial imagery processing in disaster assessment requires exceptional accuracy, computational efficiency, and real-time processing capabilities. Traditionally Convolutional Neural Networks (CNNs) demonstrate efficiency in local feature extraction but are limited by their potential for global context interpretation. Vision Transformers (ViTs) show promise for improved global context interpretation through the use of attention mechanisms, although they still remain underinvestigated in UAV-based disaster response applications.
  arXiv  Detail & Related papers  (2024-10-17T15:25:13Z)
• UAVDB: Point-Guided Masks for UAV Detection and Segmentation [0.03464344220266879]
  We present UAVDB, a new benchmark dataset for UAV detection and segmentation.<n>It is built upon a point-guided weak supervision pipeline.<n>UAVDB captures UAVs at diverse scales, from visible objects to near-single-pixel instances.
  arXiv  Detail & Related papers  (2024-09-09T13:27:53Z)
• Tiny Multi-Agent DRL for Twins Migration in UAV Metaverses: A Multi-Leader Multi-Follower Stackelberg Game Approach [57.15309977293297]
  The synergy between Unmanned Aerial Vehicles (UAVs) and metaverses is giving rise to an emerging paradigm named UAV metaverses. We propose a tiny machine learning-based Stackelberg game framework based on pruning techniques for efficient UT migration in UAV metaverses.
  arXiv  Detail & Related papers  (2024-01-18T02:14:13Z)
• Evidential Detection and Tracking Collaboration: New Problem, Benchmark and Algorithm for Robust Anti-UAV System [56.51247807483176]
  Unmanned Aerial Vehicles (UAVs) have been widely used in many areas, including transportation, surveillance, and military. Previous works have simplified such an anti-UAV task as a tracking problem, where prior information of UAVs is always provided. In this paper, we first formulate a new and practical anti-UAV problem featuring the UAVs perception in complex scenes without prior UAVs information.
  arXiv  Detail & Related papers  (2023-06-27T19:30:23Z)
• Integrated Sensing, Computation, and Communication for UAV-assisted Federated Edge Learning [52.7230652428711]
  Federated edge learning (FEEL) enables privacy-preserving model training through periodic communication between edge devices and the server. Unmanned Aerial Vehicle (UAV)mounted edge devices are particularly advantageous for FEEL due to their flexibility and mobility in efficient data collection.
  arXiv  Detail & Related papers  (2023-06-05T16:01:33Z)
• Vision-Based UAV Self-Positioning in Low-Altitude Urban Environments [20.69412701553767]
  Unmanned Aerial Vehicles (UAVs) rely on satellite systems for stable positioning. In such situations, vision-based techniques can serve as an alternative, ensuring the self-positioning capability of UAVs. This paper presents a new dataset, DenseUAV, which is the first publicly available dataset designed for the UAV self-positioning task.
  arXiv  Detail & Related papers  (2022-01-23T07:18:55Z)
• An Empirical Study of Training End-to-End Vision-and-Language Transformers [50.23532518166621]
  We present METER(textbfMultimodal textbfEnd-to-end textbfTransformtextbfER), through which we investigate how to design and pre-train a fully transformer-based VL model. Specifically, we dissect the model designs along multiple dimensions: vision encoders (e.g., CLIP-ViT, Swin transformer), text encoders (e.g., RoBERTa, DeBERTa), multimodal fusion (e.g., merged attention vs. co-
  arXiv  Detail & Related papers  (2021-11-03T17:55:36Z)
• Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking [59.06167734555191]
  Unmanned Aerial Vehicle (UAV) offers lots of applications in both commerce and recreation. We consider the task of tracking UAVs, providing rich information such as location and trajectory. We propose a dataset, Anti-UAV, with more than 300 video pairs containing over 580k manually annotated bounding boxes.
  arXiv  Detail & Related papers  (2021-01-21T07:00:15Z)
• Perceiving Traffic from Aerial Images [86.994032967469]
  We propose an object detection method called Butterfly Detector that is tailored to detect objects in aerial images. We evaluate our Butterfly Detector on two publicly available UAV datasets (UAVDT and VisDrone 2019) and show that it outperforms previous state-of-the-art methods while remaining real-time.
  arXiv  Detail & Related papers  (2020-09-16T11:37:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.