Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration

• URL: http://arxiv.org/abs/2608.10680v1
• Date: Tue, 11 Aug 2026 09:02:37 GMT
• Title: Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration
• Abstract summary: We propose an end-to-end visible-infrared oriented object detector tailored for severely cross-modal geometric discrepancies.<n>The proposed JFRDet achieves 69.7% $mathrmmAP_50$ on DroneVehicle Misaligned (DVMA), which represents state-of-the-art (SOTA) performance.
• Score: 17.74037850404483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visible-infrared object detection relies on complementary RGB and thermal cues, but its performance is often degraded by cross-modal spatial misalignment. Most existing methods rely on implicit feature adaptation to handle weakly misaligned scenarios, while large-offset geometric discrepancies remain insufficiently addressed. In this paper, we propose a Joint Feature-domain Registration and Detection network (JFRDet), an end-to-end visible-infrared oriented object detector tailored for severely cross-modal geometric discrepancies. JFRDet introduces a Cross-Modal Affine Alignment (CMAA) module to estimate an image-level affine transformation for explicit multi-level feature alignment. Note that illumination changes directly affect the reliability of RGB cues, an Illumination-Guided Complementary Fusion (IGCF) module adaptively exploits modality reliability under varying illumination conditions for cross-modal fusion. Then, an Alignment Quality-Consistency Gating (AQCG) strategy stabilizes joint optimization by modulating detection supervision according to alignment reliability and gradient consistency. We further construct DroneVehicle Misaligned (DVMA), a benchmark for evaluating visible-infrared oriented object detection under severe cross-modal geometric misalignment. The proposed JFRDet achieves 69.7\% $\mathrm{mAP}_{50}$ on DVMA, which represents state-of-the-art (SOTA) performance. The code and dataset will be available on GitHub.

Related papers

• GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception [7.90322991085067]
  Unmanned aerial vehicle (UAV) multimodal perception integrates visible (RGB), infrared (IR), synthetic aperture radar (SAR) and depth sensors for scene understanding under diverse conditions.<n>We propose GAAT (Geometry-Aware Alignment Transformer), an alignment-first pretrained model that estimates local correspondence reliability before cross-modal interaction.<n>We introduce UAVMeta and StateBench, which provide four acquisition-state scores derived from platform telemetry and image statistics.
  arXiv  Detail & Related papers  (2026-08-28T06:36:17Z)
• BC-IHV: Conditioning the Color Space for Stable Rectified-Flow Low-Light Enhancement [51.79298431282401]
  Low-light image enhancement must correct ambiguous exposure without overwriting structure already supported by the input.<n>We propose Structure-Anchored Rectified Flow (SA-RF), which maintains correspondence through separate chromaticity/intensity stems.<n>We also introduce BC-IHV, a learnable Box--Cox polar color space whose analytically invertible intensity mapping controls the inverse-gradient dynamic range through a single exponent.
  arXiv  Detail & Related papers  (2026-08-22T08:43:00Z)
• EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection [11.826446098340048]
  We propose EGM-Det, an entropy-guided adaptive fusion framework for RGB-IR object detection.<n>EGM-Det employs a dual-stream architecture to preserve modality-specific representations.<n>It selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features.
  arXiv  Detail & Related papers  (2026-08-12T05:51:48Z)
• Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking [58.338848435042536]
  RGB-Event tracking improves localization by fusing RGB appearance textures and dense temporal motion cues from event sensors.<n>Real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion.<n>We present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation.
  arXiv  Detail & Related papers  (2026-06-24T23:34:06Z)
• Spatial Orthogonal Refinement for Robust RGB-Event Visual Object Tracking [8.019596736149834]
  We propose SOR-Track, a framework for robust RGB-Event tracking based on Spatial Orthogonal Refinement (SOR)<n>Experiments on the large-scale FE108 benchmark demonstrate that SOR-Track consistently outperforms existing fusion-based trackers.<n>Despite its simplicity, the proposed method offers a principled and physics-grounded approach to multi-modal feature alignment and texture rectification.
  arXiv  Detail & Related papers  (2026-03-29T23:54:31Z)
• RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection [76.1829298714382]
  This paper focuses on the inconsistency in salient regions between RGB and thermal images.<n>We propose the Region-guided Selective Optimization Network for RGB-T Salient Object Detection.<n>We conduct extensive experiments on the RGB-T dataset, and the results demonstrate that the proposed RSONet achieves competitive performance against 27 state-of-the-art SOD methods.
  arXiv  Detail & Related papers  (2026-03-13T06:01:17Z)
• RTFDNet: Fusion-Decoupling for Robust RGB-T Segmentation [0.2578242050187029]
  RGB-Thermal (RGB-T) semantic segmentation is essential for robotic systems operating in low-light environments.<n>Traditional approaches often overemphasize modality balance, resulting in limited robustness or severe performance when sensor signals are partially missing.
  arXiv  Detail & Related papers  (2026-03-10T03:40:26Z)
• Cross-Modal Alignment and Fusion for RGB-D Transmission-Line Defect Detection [11.637942429146172]
  This paper proposes CMAFNet, a Cross-Modal Alignment and Fusion Network that integrates RGB appearance and depth geometry through a principled-then-fuse paradigm.<n> CMAFNet consists of a Semantic Recomposition Module that performs dictionary-based feature purification.<n>A lightweight variant reaches 24.8% mAP50 at 228 FPS with only 4.9M parameters, surpassing all YOLO-based detectors.
  arXiv  Detail & Related papers  (2026-02-02T06:11:33Z)
• Multi-Domain Biometric Recognition using Body Embeddings [51.36007967653781]
  We show that body embeddings perform better than face embeddings in medium-wave infrared (MWIR) and long-wave infrared (LWIR) domains.<n>We leverage a vision transformer architecture to establish benchmark results on the IJB-MDF dataset.<n>We also show that finetuning a body model, pretrained exclusively on VIS data, with a simple combination of cross-entropy and triplet losses achieves state-of-the-art mAP scores.
  arXiv  Detail & Related papers  (2025-03-13T22:38:18Z)
• Bringing RGB and IR Together: Hierarchical Multi-Modal Enhancement for Robust Transmission Line Detection [67.02804741856512]
  We propose a novel Hierarchical Multi-Modal Enhancement Network (HMMEN) that integrates RGB and IR data for robust and accurate TL detection.<n>Our method introduces two key components: (1) a Mutual Multi-Modal Enhanced Block (MMEB), which fuses and enhances hierarchical RGB and IR feature maps in a coarse-to-fine manner, and (2) a Feature Alignment Block (FAB) that corrects misalignments between decoder outputs and IR feature maps by leveraging deformable convolutions.
  arXiv  Detail & Related papers  (2025-01-25T06:21:06Z)
• DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection [57.08921921586688]
  Infrared-visible object detection aims to achieve robust object detection by leveraging the complementary information of infrared and visible image pairs.<n> fusing misalignment complementary features is difficult, and current methods cannot reliably locate objects in both modalities under misalignment conditions.<n>We propose a Decoupled Position Detection Transformer to address these issues.<n> Experiments on DroneVehicle and KAIST datasets demonstrate significant improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-12T13:05:43Z)
• RGB-D Salient Object Detection with Cross-Modality Modulation and Selection [126.4462739820643]
  We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD) The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features.
  arXiv  Detail & Related papers  (2020-07-14T14:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.