Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion

• URL: http://arxiv.org/abs/2505.06920v1
• Date: Sun, 11 May 2025 09:36:25 GMT
• Title: Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion
• Authors: Timing Li, Bing Cao, Pengfei Zhu, Bin Xiao, Qinghua Hu,
• Abstract summary: Acquiring accurately aligned multi-modal image pairs is fundamental for achieving high-quality multi-modal image fusion.<n>We propose a novel self-supervised textbfBi-directional textbfSelf-textbfRegistration framework (textbfB-SR)
• Score: 53.60977801655896
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Acquiring accurately aligned multi-modal image pairs is fundamental for achieving high-quality multi-modal image fusion. To address the lack of ground truth in current multi-modal image registration and fusion methods, we propose a novel self-supervised \textbf{B}i-directional \textbf{S}elf-\textbf{R}egistration framework (\textbf{B-SR}). Specifically, B-SR utilizes a proxy data generator (PDG) and an inverse proxy data generator (IPDG) to achieve self-supervised global-local registration. Visible-infrared image pairs with spatially misaligned differences are aligned to obtain global differences through the registration module. The same image pairs are processed by PDG, such as cropping, flipping, stitching, etc., and then aligned to obtain local differences. IPDG converts the obtained local differences into pseudo-global differences, which are used to perform global-local difference consistency with the global differences. Furthermore, aiming at eliminating the effect of modal gaps on the registration module, we design a neighborhood dynamic alignment loss to achieve cross-modal image edge alignment. Extensive experiments on misaligned multi-modal images demonstrate the effectiveness of the proposed method in multi-modal image alignment and fusion against the competing methods. Our code will be publicly available.

Related papers

• OSDM-MReg: Multimodal Image Registration based One Step Diffusion Model [8.619958921346184]
  Multimodal remote sensing image registration aligns images from different sensors for data fusion and analysis.<n>We propose OSDM-MReg, a novel multimodal image registration framework based image-to-image translation.<n> Experiments demonstrate superior accuracy and efficiency across various multimodal registration tasks.
  arXiv  Detail & Related papers  (2025-04-08T13:32:56Z)
• From Cross-Modal to Mixed-Modal Visible-Infrared Re-Identification [11.324518300593983]
  Current VI-ReID methods focus on cross-modality matching, but real-world applications often involve mixed galleries containing both V and I images.<n>This is because gallery images from the same modality may have lower domain gaps but correspond to different identities.<n>This paper introduces a novel mixed-modal ReID setting, where galleries contain data from both modalities.
  arXiv  Detail & Related papers  (2025-01-23T01:28:05Z)
• BSAFusion: A Bidirectional Stepwise Feature Alignment Network for Unaligned Medical Image Fusion [11.306367018981678]
  This paper proposes an unaligned medical image fusion method called Bidirectional Stepwise Feature Alignment and Fusion.<n>In terms of feature alignment, BSFA-F employs a bidirectional stepwise alignment deformation field prediction strategy.<n>The experimental results across multiple datasets demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-12-11T02:56:23Z)
• Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation [109.1912721224697]
  We present the Unified Frequency-Assisted transFormer framework, named UFAFormer, to address the DGM4 problem. By leveraging the discrete wavelet transform, we decompose images into several frequency sub-bands, capturing rich face forgery artifacts. Our proposed frequency encoder, incorporating intra-band and inter-band self-attentions, explicitly aggregates forgery features within and across diverse sub-bands.
  arXiv  Detail & Related papers  (2023-09-18T11:06:42Z)
• Mutual-Guided Dynamic Network for Image Fusion [51.615598671899335]
  We propose a novel mutual-guided dynamic network (MGDN) for image fusion, which allows for effective information utilization across different locations and inputs. Experimental results on five benchmark datasets demonstrate that our proposed method outperforms existing methods on four image fusion tasks.
  arXiv  Detail & Related papers  (2023-08-24T03:50:37Z)
• Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration [67.23451452670282]
  Misalignments between multi-modality images pose challenges in image fusion. We propose a Cross-modality Multi-scale Progressive Dense Registration scheme. This scheme accomplishes the coarse-to-fine registration exclusively using a one-stage optimization.
  arXiv  Detail & Related papers  (2023-08-22T03:46:24Z)
• Dual-Flow Transformation Network for Deformable Image Registration with Region Consistency Constraint [95.30864269428808]
  Current deep learning (DL)-based image registration approaches learn the spatial transformation from one image to another by leveraging a convolutional neural network. We present a novel dual-flow transformation network with region consistency constraint which maximizes the similarity of ROIs within a pair of images. Experiments on four public 3D MRI datasets show that the proposed method achieves the best registration performance in accuracy and generalization.
  arXiv  Detail & Related papers  (2021-12-04T05:30:44Z)
• Global and Local Alignment Networks for Unpaired Image-to-Image Translation [170.08142745705575]
  The goal of unpaired image-to-image translation is to produce an output image reflecting the target domain's style. Due to the lack of attention to the content change in existing methods, semantic information from source images suffers from degradation during translation. We introduce a novel approach, Global and Local Alignment Networks (GLA-Net) Our method effectively generates sharper and more realistic images than existing approaches.
  arXiv  Detail & Related papers  (2021-11-19T18:01:54Z)
• DF^2AM: Dual-level Feature Fusion and Affinity Modeling for RGB-Infrared Cross-modality Person Re-identification [18.152310122348393]
  RGB-infrared person re-identification is a challenging task due to the intra-class variations and cross-modality discrepancy. We propose a Dual-level (i.e., local and global) Feature Fusion (DF2) module by learning attention for discnative feature from local to global manner. To further mining the relationships between global features from person images, we propose an Affinities Modeling (AM) module.
  arXiv  Detail & Related papers  (2021-04-01T03:12:56Z)
• CoMIR: Contrastive Multimodal Image Representation for Registration [4.543268895439618]
  We propose contrastive coding to learn shared, dense image representations, referred to as CoMIRs (Contrastive Multimodal Image Representations) CoMIRs enable the registration of multimodal images where existing registration methods often fail due to a lack of sufficiently similar image structures.
  arXiv  Detail & Related papers  (2020-06-11T10:51:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.