FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection

• URL: http://arxiv.org/abs/2412.09258v1
• Date: Thu, 12 Dec 2024 13:19:05 GMT
• Title: FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection
• Authors: Ke Li, Di Wang, Zhangyuan Hu, Shaofeng Li, Weiping Ni, Lin Zhao, Quan Wang,
• Abstract summary: Infrared-visible object detection (IVOD) seeks to harness the complementary information in infrared and visible images. Existing methods often neglect the frequency characteristics of complementary information. We introduce a novel Frequency-Driven Feature Decomposition Network for IVOD, called FD2-Net.
• Score: 16.80713499383237
• License:
• Abstract: Infrared-visible object detection (IVOD) seeks to harness the complementary information in infrared and visible images, thereby enhancing the performance of detectors in complex environments. However, existing methods often neglect the frequency characteristics of complementary information, such as the abundant high-frequency details in visible images and the valuable low-frequency thermal information in infrared images, thus constraining detection performance. To solve this problem, we introduce a novel Frequency-Driven Feature Decomposition Network for IVOD, called FD2-Net, which effectively captures the unique frequency representations of complementary information across multimodal visual spaces. Specifically, we propose a feature decomposition encoder, wherein the high-frequency unit (HFU) utilizes discrete cosine transform to capture representative high-frequency features, while the low-frequency unit (LFU) employs dynamic receptive fields to model the multi-scale context of diverse objects. Next, we adopt a parameter-free complementary strengths strategy to enhance multimodal features through seamless inter-frequency recoupling. Furthermore, we innovatively design a multimodal reconstruction mechanism that recovers image details lost during feature extraction, further leveraging the complementary information from infrared and visible images to enhance overall representational capacity. Extensive experiments demonstrate that FD2-Net outperforms state-of-the-art (SOTA) models across various IVOD benchmarks, i.e. LLVIP (96.2% mAP), FLIR (82.9% mAP), and M3FD (83.5% mAP).

Related papers

• FE-UNet: Frequency Domain Enhanced U-Net with Segment Anything Capability for Versatile Image Segmentation [50.9040167152168]
  We experimentally quantify the contrast sensitivity function of CNNs and compare it with that of the human visual system. We propose the Wavelet-Guided Spectral Pooling Module (WSPM) to enhance and balance image features across the frequency domain. To further emulate the human visual system, we introduce the Frequency Domain Enhanced Receptive Field Block (FE-RFB) We develop FE-UNet, a model that utilizes SAM2 as its backbone and incorporates Hiera-Large as a pre-trained block.
  arXiv  Detail & Related papers  (2025-02-06T07:24:34Z)
• Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution [54.293362972473595]
  Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Current approaches to address SR tasks are either dedicated to extracting RGB image features or assuming similar degradation patterns. We propose a Contourlet refinement gate framework to restore infrared modal-specific features while preserving spectral distribution fidelity.
  arXiv  Detail & Related papers  (2024-11-19T14:24:03Z)
• SFDFusion: An Efficient Spatial-Frequency Domain Fusion Network for Infrared and Visible Image Fusion [11.46957526079837]
  Infrared and visible image fusion aims to generate fused images with prominent targets and rich texture details. This paper proposes an efficient Spatial-Frequency Domain Fusion network for infrared and visible image fusion. Our method produces fused images with significant advantages in various fusion metrics and visual effects.
  arXiv  Detail & Related papers  (2024-10-30T09:17:23Z)
• Wavelet-based Bi-dimensional Aggregation Network for SAR Image Change Detection [53.842568573251214]
  Experimental results on three SAR datasets demonstrate that our WBANet significantly outperforms contemporary state-of-the-art methods. Our WBANet achieves 98.33%, 96.65%, and 96.62% of percentage of correct classification (PCC) on the respective datasets.
  arXiv  Detail & Related papers  (2024-07-18T04:36:10Z)
• Fusion of Infrared and Visible Images based on Spatial-Channel Attentional Mechanism [3.388001684915793]
  We present AMFusionNet, an innovative approach to infrared and visible image fusion (IVIF) By assimilating thermal details from infrared images with texture features from visible sources, our method produces images enriched with comprehensive information. Our method outperforms state-of-the-art algorithms in terms of quality and quantity.
  arXiv  Detail & Related papers  (2023-08-25T21:05:11Z)
• Complementary Frequency-Varying Awareness Network for Open-Set Fine-Grained Image Recognition [14.450381668547259]
  Open-set image recognition is a challenging topic in computer vision. We propose a Complementary Frequency-varying Awareness Network that could better capture both high-frequency and low-frequency information. Based on CFAN, we propose an open-set fine-grained image recognition method, called CFAN-OSFGR.
  arXiv  Detail & Related papers  (2023-07-14T08:15:36Z)
• A Scale-Arbitrary Image Super-Resolution Network Using Frequency-domain Information [42.55177009667711]
  Image super-resolution (SR) is a technique to recover lost high-frequency information in low-resolution (LR) images. In this paper, we study image features in the frequency domain to design a novel scale-arbitrary image SR network.
  arXiv  Detail & Related papers  (2022-12-08T15:10:49Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection [65.30079184700755]
  This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks. This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network.
  arXiv  Detail & Related papers  (2022-03-30T11:44:56Z)
• Wavelet-Based Network For High Dynamic Range Imaging [64.66969585951207]
  Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal. In this work, we propose a novel frequency-guided end-to-end deep neural network (FNet) to conduct HDR fusion in the frequency domain, and Wavelet Transform (DWT) is used to decompose inputs into different frequency bands. The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details.
  arXiv  Detail & Related papers  (2021-08-03T12:26:33Z)
• Thermal Image Super-Resolution Using Second-Order Channel Attention with Varying Receptive Fields [4.991042925292453]
  We introduce a system to efficiently reconstruct thermal images. The restoration of thermal images is critical for applications that involve safety, search and rescue, and military operations.
  arXiv  Detail & Related papers  (2021-07-30T22:17:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.