EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection

• URL: http://arxiv.org/abs/2608.11685v1
• Date: Wed, 12 Aug 2026 05:51:48 GMT
• Title: EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection
• Abstract summary: We propose EGM-Det, an entropy-guided adaptive fusion framework for RGB-IR object detection.<n>EGM-Det employs a dual-stream architecture to preserve modality-specific representations.<n>It selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features.
• Score: 11.826446098340048
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.

Related papers

• P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation [73.147281449267]
  We propose P2Fusion, a prior-guided distillation-based framework that reformulates infrared-visible image fusion.<n>Instead of imposing hard-coded penalties, we distill image-intrinsic priors, thermal saliency and spatial quality, into learnable dynamic regulators.<n>P2Fusion achieves state-of-the-art results in 14 out of 20 key evaluation metrics across 5 benchmarks.
  arXiv  Detail & Related papers  (2026-08-13T10:09:49Z)
• Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration [17.74037850404483]
  We propose an end-to-end visible-infrared oriented object detector tailored for severely cross-modal geometric discrepancies.<n>The proposed JFRDet achieves 69.7% $mathrmmAP_50$ on DroneVehicle Misaligned (DVMA), which represents state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-08-11T09:02:37Z)
• Frequency-Guided Fusion For RGB-Thermal Semantic Segmentation [0.6187780920448871]
  We propose a multi-modal fusion architecture built upon dual ConvNeXt V2 backbones that employs stage-wise, modality-adaptive fusion strategies.<n>Experiments on MFNet and PST900 demonstrate that our lightest variant achieves 61.73% and 86.24% mIoU, respectively, with only 35.43M parameters.
  arXiv  Detail & Related papers  (2026-05-25T18:57:22Z)
• Tri-Modal Fusion Transformers for UAV-based Object Detection [1.338174941551702]
  We present a tri-modal framework that processes RGB, thermal, and event data with a dual-stream hierarchical vision transformer.<n>We introduce a 10,489-frame UAV dataset with synchronized and pre-aligned RGB-thermal-event streams and 24,223 annotated vehicles.
  arXiv  Detail & Related papers  (2026-04-17T18:35:06Z)
• RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection [76.1829298714382]
  This paper focuses on the inconsistency in salient regions between RGB and thermal images.<n>We propose the Region-guided Selective Optimization Network for RGB-T Salient Object Detection.<n>We conduct extensive experiments on the RGB-T dataset, and the results demonstrate that the proposed RSONet achieves competitive performance against 27 state-of-the-art SOD methods.
  arXiv  Detail & Related papers  (2026-03-13T06:01:17Z)
• HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection [75.406055413928]
  We propose a novel prompt-driven segment anything model (HyPSAM) for RGB-T SOD.<n> DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction.<n>Experiments on three public datasets demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-23T07:32:11Z)
• Multispectral Detection Transformer with Infrared-Centric Feature Fusion [8.762314897895175]
  Infrared-Centric Fusion (IC-Fusion) is a lightweight and modality-aware sensor fusion method.<n>IC-Fusion prioritizes infrared features while effectively integrating complementary RGB semantic context.<n> Experiments on the FLIR and LLVIP benchmarks demonstrate the superior effectiveness and efficiency of our IR-centric fusion strategy.
  arXiv  Detail & Related papers  (2025-05-21T05:44:14Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• Transformer-based Network for RGB-D Saliency Detection [82.6665619584628]
  Key to RGB-D saliency detection is to fully mine and fuse information at multiple scales across the two modalities. We show that transformer is a uniform operation which presents great efficacy in both feature fusion and feature enhancement. Our proposed network performs favorably against state-of-the-art RGB-D saliency detection methods.
  arXiv  Detail & Related papers  (2021-12-01T15:53:58Z)
• EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation [62.210091681352914]
  We study multi-sensor fusion for 3D semantic segmentation for many applications, such as autonomous driving and robotics. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) We propose a two-stream network to extract features from the two modalities separately. The extracted features are fused by effective residual-based fusion modules.
  arXiv  Detail & Related papers  (2021-06-21T10:47:26Z)
• RGB-D Salient Object Detection with Cross-Modality Modulation and Selection [126.4462739820643]
  We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD) The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features.
  arXiv  Detail & Related papers  (2020-07-14T14:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.