Alignment-Aware and Reliability-Gated Multimodal Fusion for Unmanned Aerial Vehicle Detection Across Heterogeneous Thermal-Visual Sensors

• URL: http://arxiv.org/abs/2603.08208v1
• Date: Mon, 09 Mar 2026 10:39:26 GMT
• Title: Alignment-Aware and Reliability-Gated Multimodal Fusion for Unmanned Aerial Vehicle Detection Across Heterogeneous Thermal-Visual Sensors
• Abstract summary: This study introduces two fusion strategies, Registration-aware Guided Image Fusion (RGIF) and Reliability-Gated Modality-Attention Fusion (RGMAF)<n>RGIF employs Enhanced Correlation Coefficient (ECC)-based affine registration combined with guided filtering to maintain thermal saliency while enhancing structural detail.<n>RGMAF integrates affine and optical-flow registration with a reliability-weighted attention mechanism that adaptively balances thermal contrast and visual sharpness.
• Score: 6.686898063543218
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reliable unmanned aerial vehicle (UAV) detection is critical for autonomous airspace monitoring but remains challenging when integrating sensor streams that differ substantially in resolution, perspective, and field of view. Conventional fusion methods-such as wavelet-, Laplacian-, and decision-level approaches-often fail to preserve spatial correspondence across modalities and suffer from annotation of inconsistencies, limiting their robustness in real-world settings. This study introduces two fusion strategies, Registration-aware Guided Image Fusion (RGIF) and Reliability-Gated Modality-Attention Fusion (RGMAF), designed to overcome these limitations. RGIF employs Enhanced Correlation Coefficient (ECC)-based affine registration combined with guided filtering to maintain thermal saliency while enhancing structural detail. RGMAF integrates affine and optical-flow registration with a reliability-weighted attention mechanism that adaptively balances thermal contrast and visual sharpness. Experiments were conducted on the Multi-Sensor and Multi-View Fixed-Wing (MMFW)-UAV dataset comprising 147,417 annotated air-to-air frames collected from infrared, wide-angle, and zoom sensors. Among single-modality detectors, YOLOv10x demonstrated the most stable cross-domain performance and was selected as the detection backbone for evaluating fused imagery. RGIF improved the visual baseline by 2.13% mAP@50 (achieving 97.65%), while RGMAF attained the highest recall of 98.64%. These findings show that registration-aware and reliability-adaptive fusion provides a robust framework for integrating heterogeneous modalities, substantially enhancing UAV detection performance in multimodal environments.

Related papers

• EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection [11.826446098340048]
  We propose EGM-Det, an entropy-guided adaptive fusion framework for RGB-IR object detection.<n>EGM-Det employs a dual-stream architecture to preserve modality-specific representations.<n>It selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features.
  arXiv  Detail & Related papers  (2026-08-12T05:51:48Z)
• Transformer-based Multisensor Data Fusion of Ultrasonic Guided Wave and FBG-based Strain Measurements for Multitask Aerospace Structural Health Monitoring [5.884291793763452]
  This paper proposes a Transformer-based data fusion framework that integrates multisensor data streams from piezoelectric transducer (PZT)<n>By an attention-mechanism visualization, the proposed framework enables transparent, multitask learning for both health indicator (HI) prediction and damage localization.
  arXiv  Detail & Related papers  (2026-06-24T13:47:27Z)
• FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales [65.4821703903285]
  FLORO is a multimodal foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus.<n> FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and UAV-derived data.<n>We evaluated FLORO on the PANGAEA benchmark under a frozen-encoder protocol across scene classification, segmentation, and regression tasks.
  arXiv  Detail & Related papers  (2026-05-27T08:55:54Z)
• Physically-Induced Atmospheric Adversarial Perturbations: Enhancing Transferability and Robustness in Remote Sensing Image Classification [54.952203312050564]
  Adversarial attacks pose a severe threat to the reliability of deep learning models in remote sensing (RS) image classification.<n>We propose FogFool, a physically plausible adversarial framework that generates fog-based perturbations.<n>Experiments on two benchmark RS datasets demonstrate that FogFool achieves superior performance.
  arXiv  Detail & Related papers  (2026-04-16T05:47:57Z)
• Perceptual Region-Driven Infrared-Visible Co-Fusion for Extreme Scene Enhancement [8.10747908396949]
  We propose a region perception-based fusion framework that combines multi-exposure and multi-modal imaging.<n>This framework co-fuses multi-modal and multi-exposure data, overcoming single-exposure method limitations in extreme environments.<n> Experiments conducted on both synthetic and real-world data demonstrate superior image clarity and improved performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-12-06T11:17:35Z)
• Transferable Dual-Domain Feature Importance Attack against AI-Generated Image Detector [32.543253278021446]
  Recent AI-generated image (AIGI) detectors achieve impressive accuracy under clean condition.<n>It is significant to develop advanced adversarial attacks for evaluating the security of such detectors.<n>This letter proposes a Dual-domain Feature Importance Attack scheme to invalidate AIGI detectors to some extent.
  arXiv  Detail & Related papers  (2025-11-19T16:03:15Z)
• A Multimodal Transformer Approach for UAV Detection and Aerial Object Recognition Using Radar, Audio, and Video Data [0.3093890460224435]
  Unmanned aerial vehicle (UAV) detection and aerial object recognition are critical for modern surveillance and security.<n>This research addresses these challenges by designing and rigorously evaluating a novel multimodal Transformer model.<n>It integrates diverse data streams: radar, visual band video (RGB), infrared (IR) video, and audio.
  arXiv  Detail & Related papers  (2025-11-19T10:22:29Z)
• Model-Agnostic Open-Set Air-to-Air Visual Object Detection for Reliable UAV Perception [7.300229659237879]
  Traditional closed-set detectors degrade significantly under domain shifts and flight data corruption.<n>We propose a novel, model-agnostic open-set detection framework designed specifically for embedding-based detectors.
  arXiv  Detail & Related papers  (2025-09-11T09:40:06Z)
• SGDFuse: SAM-Guided Diffusion for High-Fidelity Infrared and Visible Image Fusion [65.80051636480836]
  This paper proposes a conditional diffusion model guided by the Segment Anything Model (SAM) to achieve high-fidelity and semantically-aware image fusion.<n>The framework operates in a two-stage process: it first performs a preliminary fusion of multi-modal features, and then utilizes the semantic masks as a condition to drive the diffusion model's coarse-to-fine denoising generation.<n>Extensive experiments demonstrate that SGDFuse achieves state-of-the-art performance in both subjective and objective evaluations.
  arXiv  Detail & Related papers  (2025-08-07T10:58:52Z)
• Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection [0.0]
  Unmanned aerial vehicle (UAV) object detection plays a vital role in applications such as environmental monitoring and urban security.<n>Due to UAV platform motion and asynchronous imaging, spatial misalignment frequently occurs between modalities, leading to weak alignment.<n>We propose Cross-modal Offset-guided Dynamic Alignment and Fusion (CoDAF) to address these issues.
  arXiv  Detail & Related papers  (2025-06-20T04:11:39Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Log-Likelihood Score Level Fusion for Improved Cross-Sensor Smartphone Periocular Recognition [52.15994166413364]
  We employ fusion of several comparators to improve periocular performance when images from different smartphones are compared. We use a probabilistic fusion framework based on linear logistic regression, in which fused scores tend to be log-likelihood ratios. Our framework also provides an elegant and simple solution to handle signals from different devices, since same-sensor and cross-sensor score distributions are aligned and mapped to a common probabilistic domain.
  arXiv  Detail & Related papers  (2023-11-02T13:43:44Z)
• Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection [65.30079184700755]
  This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks. This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network.
  arXiv  Detail & Related papers  (2022-03-30T11:44:56Z)
• ASFD: Automatic and Scalable Face Detector [59.31799101216593]
  We propose to search an effective FAE architecture, termed AutoFAE, which outperforms all existing FAE modules in face detection with a considerable margin. In particular, our strong ASFD-D6 outperforms the best competitor with AP 96.7/96.2/92.1 on WIDER Face test, and the lightweight ASFD-D0 costs about 3.1 ms, more than 320 FPS.
  arXiv  Detail & Related papers  (2022-01-26T07:11:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.