Reusing Fusion-Time Spectral Reliability for Adaptive Fusion and Expert Routing in RGB-Infrared Object Detection

• URL: http://arxiv.org/abs/2606.01173v1
• Date: Sun, 31 May 2026 11:28:41 GMT
• Title: Reusing Fusion-Time Spectral Reliability for Adaptive Fusion and Expert Routing in RGB-Infrared Object Detection
• Abstract summary: We propose to extract a parameter-free, 7-dimensional spectral reliability descriptor.<n>Describes band energy, amplitude ratio, phase consistency, and cross-modal correlation.<n>We show that preserving fusion-time reliability as an explicit signal benefits both adaptive fusion and post-fusion conditional computation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: RGB-infrared detectors typically discard the statistics generated during cross-modal fusion, leaving downstream modules unaware of whether the current interaction is reliable. We propose to extract a parameter-free, 7-dimensional spectral reliability descriptor -- summarizing band energy, amplitude ratio, phase consistency, and cross-modal correlation -- and to reuse it beyond the fusion stage. The descriptor drives both Spectral Reliability Fusion (SRF), which gates a spectral residual against a conservative spatial base, and Reliability-Conditioned Expert Routing (RCER), which combines the descriptor with pooled content to steer sparse post-fusion experts. Under matched ablations, descriptor-aware gating improves mAP50 over content-only adaptive gating; a $2{\times}2$ factorial analysis further shows that descriptor-conditioned routing provides the larger marginal gain over expert architecture alone at near-equal parameter count. Under six synthetic degradations on DroneVehicle, average retention rises to 95.0%, versus 92.0% for content-only MoE and 87.9% for concatenation, with the largest gain under modality drop; the same model also improves mAP50 by +5.2/+5.3 on the natural day/night split. These results suggest that preserving fusion-time reliability as an explicit signal benefits both adaptive fusion and post-fusion conditional computation.

Related papers

• S$^3$AM: A Single-Stream SAM with Reliability-Calibrated Frequency Adapter for Multi-modal Salient Object Detection [74.15663758681849]
  Single-stream framework integrates reliability-calibrated frequency adaptation into the adopted SAM backbone for MSOD.<n>Experiments on RGB-D, RGB-T, and RGB-NIR salient object detection benchmarks validate that the proposed framework achieves competitive performance.
  arXiv  Detail & Related papers  (2026-08-18T08:05:00Z)
• P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation [73.147281449267]
  We propose P2Fusion, a prior-guided distillation-based framework that reformulates infrared-visible image fusion.<n>Instead of imposing hard-coded penalties, we distill image-intrinsic priors, thermal saliency and spatial quality, into learnable dynamic regulators.<n>P2Fusion achieves state-of-the-art results in 14 out of 20 key evaluation metrics across 5 benchmarks.
  arXiv  Detail & Related papers  (2026-08-13T10:09:49Z)
• Dual-Branch Gated Fusion for Open-Set Audio Deepfake Source Tracing [2.657233098224094]
  We propose a dual-branch gated fusion framework that pairs XLSR-53 with CORES, a 66-dimensional descriptor.<n>On the MLAAD benchmark, our system achieves 97.6% ID accuracy, 4.9% EERc, and an 83.5% relative FPR95 reduction over the Interspeech 2025 baseline.
  arXiv  Detail & Related papers  (2026-06-08T22:22:48Z)
• Frequency-Guided Fusion For RGB-Thermal Semantic Segmentation [0.6187780920448871]
  We propose a multi-modal fusion architecture built upon dual ConvNeXt V2 backbones that employs stage-wise, modality-adaptive fusion strategies.<n>Experiments on MFNet and PST900 demonstrate that our lightest variant achieves 61.73% and 86.24% mIoU, respectively, with only 35.43M parameters.
  arXiv  Detail & Related papers  (2026-05-25T18:57:22Z)
• LER-YOLO: Reliability-Aware Expert Routing for Misaligned RGB-Infrared UAV Detection [6.816296803605649]
  We propose LER-YOLO, a reliability-aware sparse mixture-of-experts framework for RGB-infrared UAV detection.<n>Experiments on the public MBU benchmark under a YOLOv5s-family protocol show that LER-YOLO achieves 89.7 +/-0.2% AP50 over three independent seeds, with a best result of 89.9%.
  arXiv  Detail & Related papers  (2026-05-20T03:33:13Z)
• RSGMamba: Reliability-Aware Self-Gated State Space Model for Multimodal Semantic Segmentation [41.11292233905035]
  Cross-modal semantic segmentation has emerged as a powerful paradigm for enhancing scene understanding.<n>In this paper, we propose a novel framework termed the Reliability-aware Self-Gated State Space Model (RSGMamba)<n>At the core of our method is the Reliability-aware Self-Gated Mamba Block (RSGMB), which explicitly models modality reliability and dynamically regulates cross-modal interactions.
  arXiv  Detail & Related papers  (2026-04-14T05:51:15Z)
• Alignment-Aware and Reliability-Gated Multimodal Fusion for Unmanned Aerial Vehicle Detection Across Heterogeneous Thermal-Visual Sensors [6.686898063543218]
  This study introduces two fusion strategies, Registration-aware Guided Image Fusion (RGIF) and Reliability-Gated Modality-Attention Fusion (RGMAF)<n>RGIF employs Enhanced Correlation Coefficient (ECC)-based affine registration combined with guided filtering to maintain thermal saliency while enhancing structural detail.<n>RGMAF integrates affine and optical-flow registration with a reliability-weighted attention mechanism that adaptively balances thermal contrast and visual sharpness.
  arXiv  Detail & Related papers  (2026-03-09T10:39:26Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Spectral Gating Networks [65.9496901693099]
  We introduce Spectral Gating Networks (SGN) to introduce frequency-rich expressivity in feed-forward networks.<n>SGN augments a standard activation pathway with a compact spectral pathway and learnable gates that allow the model to start from a stable base behavior.<n>It consistently improves accuracy-efficiency trade-offs under comparable computational budgets.
  arXiv  Detail & Related papers  (2026-02-07T20:00:49Z)
• PAD: Phase-Amplitude Decoupling Fusion for Multi-Modal Land Cover Classification [49.37555541088792]
  Phase-Amplitude Decoupling (PAD) is a frequency-aware framework that separates phase (modality-shared) and amplitude (modality-complementary) components.<n>This work establishes a new paradigm for physics-aware multi-modal fusion in remote sensing.
  arXiv  Detail & Related papers  (2025-04-27T07:21:42Z)
• Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems [89.35169042718739]
  collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers.<n>Recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs)<n>This work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA.<n>Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy algorithm to enhance the inversion robustness
  arXiv  Detail & Related papers  (2025-03-01T07:15:21Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.