Causality-Driven Infrared and Visible Image Fusion

• URL: http://arxiv.org/abs/2505.20830v1
• Date: Tue, 27 May 2025 07:48:52 GMT
• Title: Causality-Driven Infrared and Visible Image Fusion
• Authors: Linli Ma, Suzhen Lin, Jianchao Zeng, Zanxia Jin, Yanbo Wang, Fengyuan Li, Yubing Luo,
• Abstract summary: This paper re-examines the image fusion task from the causality perspective.<n>It disentangles the model from the impact of bias by constructing a tailored causal graph.<n>Back-door Adjustment based Feature Fusion Module (BAFFM) is proposed to eliminate confounder interference.
• Score: 7.454657847653563
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image fusion aims to combine complementary information from multiple source images to generate more comprehensive scene representations. Existing methods primarily rely on the stacking and design of network architectures to enhance the fusion performance, often ignoring the impact of dataset scene bias on model training. This oversight leads the model to learn spurious correlations between specific scenes and fusion weights under conventional likelihood estimation framework, thereby limiting fusion performance. To solve the above problems, this paper first re-examines the image fusion task from the causality perspective, and disentangles the model from the impact of bias by constructing a tailored causal graph to clarify the causalities among the variables in image fusion task. Then, the Back-door Adjustment based Feature Fusion Module (BAFFM) is proposed to eliminate confounder interference and enable the model to learn the true causal effect. Finally, Extensive experiments on three standard datasets prove that the proposed method significantly surpasses state-of-the-art methods in infrared and visible image fusion.

Related papers

• PIF-Net: Ill-Posed Prior Guided Multispectral and Hyperspectral Image Fusion via Invertible Mamba and Fusion-Aware LoRA [0.16385815610837165]
  The goal of multispectral and hyperspectral image fusion (MHIF) is to generate high-quality images that simultaneously possess rich spectral information and fine spatial details.<n>Previous studies have not effectively addressed the ill-posed nature caused by data misalignment.<n>We propose a fusion framework named PIF-Net, which explicitly incorporates ill-posed priors to effectively fuse multispectral images and hyperspectral images.
  arXiv  Detail & Related papers  (2025-08-01T09:17:17Z)
• DFVO: Learning Darkness-free Visible and Infrared Image Disentanglement and Fusion All at Once [57.15043822199561]
  A Darkness-Free network is proposed to handle Visible and infrared image disentanglement and fusion all at Once (DFVO)<n>DFVO employs a cascaded multi-task approach to replace the traditional two-stage cascaded training (enhancement and fusion)<n>Our proposed approach outperforms state-of-the-art alternatives in terms of qualitative and quantitative evaluations.
  arXiv  Detail & Related papers  (2025-05-07T15:59:45Z)
• Guided and Variance-Corrected Fusion with One-shot Style Alignment for Large-Content Image Generation [2.3141583665677503]
  A common approach involves jointly generating a series of overlapped image patches and obtaining large images by merging adjacent patches.<n>Results from existing methods often exhibit noticeable artifacts, e.g., seams and inconsistent objects and styles.<n>We propose Guided Fusion (GF), which mitigates the negative impact from distant image regions by applying a weighted average to the overlapping regions.<n>We also propose Variance-Corrected Fusion (VCF), which corrects data variance at post-averaging, generating more accurate fusion for the Denoising Diffusion Probabilistic Model.
  arXiv  Detail & Related papers  (2024-12-17T10:33:34Z)
• CoMoFusion: Fast and High-quality Fusion of Infrared and Visible Image with Consistency Model [20.02742423120295]
  Current generative models based fusion methods often suffer from unstable training and slow inference speed. CoMoFusion can generate the high-quality images and achieve fast image inference speed. In order to enhance the texture and salient information of fused images, a novel loss based on pixel value selection is also designed.
  arXiv  Detail & Related papers  (2024-05-31T12:35:06Z)
• Decomposition-based and Interference Perception for Infrared and Visible Image Fusion in Complex Scenes [4.919706769234434]
  We propose a decomposition-based and interference perception image fusion method. We classify the pixels of visible image from the degree of scattering of light transmission, based on which we then separate the detail and energy information of the image. This refined decomposition facilitates the proposed model in identifying more interfering pixels that are in complex scenes.
  arXiv  Detail & Related papers  (2024-02-03T09:27:33Z)
• PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation [50.556961575275345]
  We propose a perception-aware fusion framework to promote segmentation robustness in adversarial scenes. We show that our scheme substantially enhances the robustness, with gains of 15.3% mIOU, compared with advanced competitors.
  arXiv  Detail & Related papers  (2023-08-08T01:55:44Z)
• A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion [69.10255211811007]
  We present a Task-guided, Implicit-searched and Meta- generalizationd (TIM) deep model to address the image fusion problem in a challenging real-world scenario. Specifically, we propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion. Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency.
  arXiv  Detail & Related papers  (2023-05-25T08:54:08Z)
• Searching a Compact Architecture for Robust Multi-Exposure Image Fusion [55.37210629454589]
  Two major stumbling blocks hinder the development, including pixel misalignment and inefficient inference. This study introduces an architecture search-based paradigm incorporating self-alignment and detail repletion modules for robust multi-exposure image fusion. The proposed method outperforms various competitive schemes, achieving a noteworthy 3.19% improvement in PSNR for general scenarios and an impressive 23.5% enhancement in misaligned scenarios.
  arXiv  Detail & Related papers  (2023-05-20T17:01:52Z)
• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion [144.9653045465908]
  We propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM) Our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2023-03-13T04:06:42Z)
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion [68.78897015832113]
  We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion.<n>Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
  arXiv  Detail & Related papers  (2022-11-20T12:02:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.