A Joint Convolution Auto-encoder Network for Infrared and Visible Image Fusion

• URL: http://arxiv.org/abs/2201.10736v1
• Date: Wed, 26 Jan 2022 03:49:27 GMT
• Title: A Joint Convolution Auto-encoder Network for Infrared and Visible Image Fusion
• Authors: Zhancheng Zhang, Yuanhao Gao, Mengyu Xiong, Xiaoqing Luo, and Xiao-Jun Wu
• Abstract summary: We design a joint convolution auto-encoder (JCAE) network for infrared and visible image fusion. Inspired by the infrared cognition ability of crotalinae animals, we design a joint convolution auto-encoder (JCAE) network for infrared and visible image fusion.
• Score: 7.799758067671958
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background: Leaning redundant and complementary relationships is a critical step in the human visual system. Inspired by the infrared cognition ability of crotalinae animals, we design a joint convolution auto-encoder (JCAE) network for infrared and visible image fusion. Methods: Our key insight is to feed infrared and visible pair images into the network simultaneously and separate an encoder stream into two private branches and one common branch, the private branch works for complementary features learning and the common branch does for redundant features learning. We also build two fusion rules to integrate redundant and complementary features into their fused feature which are then fed into the decoder layer to produce the final fused image. We detail the structure, fusion rule and explain its multi-task loss function. Results: Our JCAE network achieves good results in terms of both subjective effect and objective evaluation metrics.

Related papers

• DAF-Net: A Dual-Branch Feature Decomposition Fusion Network with Domain Adaptive for Infrared and Visible Image Fusion [21.64382683858586]
  Infrared and visible image fusion aims to combine complementary information from both modalities to provide a more comprehensive scene understanding. We propose a dual-branch feature decomposition fusion network (DAF-Net) with Maximum domain adaptive. By incorporating MK-MMD, the DAF-Net effectively aligns the latent feature spaces of visible and infrared images, thereby improving the quality of the fused images.
  arXiv  Detail & Related papers  (2024-09-18T02:14:08Z)
• CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approach [9.253098561330978]
  Cross attention mechanism (CAM) is proposed to enhance the complementary information. Two-stage training strategy based fusion scheme is presented to generate the fused images. Experiments show that our proposed fusion method obtains the SOTA fusion performance compared with the existing fusion networks.
  arXiv  Detail & Related papers  (2024-06-15T09:52:42Z)
• MaeFuse: Transferring Omni Features with Pretrained Masked Autoencoders for Infrared and Visible Image Fusion via Guided Training [57.18758272617101]
  MaeFuse is a novel autoencoder model designed for infrared and visible image fusion (IVIF) Our model utilizes a pretrained encoder from Masked Autoencoders (MAE), which facilities the omni features extraction for low-level reconstruction and high-level vision tasks. MaeFuse not only introduces a novel perspective in the realm of fusion techniques but also stands out with impressive performance across various public datasets.
  arXiv  Detail & Related papers  (2024-04-17T02:47:39Z)
• An Interactively Reinforced Paradigm for Joint Infrared-Visible Image Fusion and Saliency Object Detection [59.02821429555375]
  This research focuses on the discovery and localization of hidden objects in the wild and serves unmanned systems. Through empirical analysis, infrared and visible image fusion (IVIF) enables hard-to-find objects apparent. multimodal salient object detection (SOD) accurately delineates the precise spatial location of objects within the picture.
  arXiv  Detail & Related papers  (2023-05-17T06:48:35Z)
• LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images [98.36300655482196]
  We formulate the fusion task mathematically, and establish a connection between its optimal solution and the network architecture that can implement it. In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model. Based on this novel network architecture, an end-to-end lightweight fusion network is constructed to fuse infrared and visible light images.
  arXiv  Detail & Related papers  (2023-04-11T12:11:23Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion [72.8898811120795]
  We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion. Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
  arXiv  Detail & Related papers  (2022-11-20T12:02:07Z)
• Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection [65.30079184700755]
  This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks. This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network.
  arXiv  Detail & Related papers  (2022-03-30T11:44:56Z)
• Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning [7.995162257955025]
  We propose a novel end-to-end mode based on generative adversarial training to achieve better fusion balance. In particular, in the generator, we construct a multi-level encoder-decoder network with a triple path, and adopt infrared and visible paths to provide additional intensity and information gradient. In addition, dual discriminators are designed to identify the similar distribution between fused result and source images, and the generator is optimized to produce a more balanced result.
  arXiv  Detail & Related papers  (2022-03-29T08:28:14Z)
• Image fusion using symmetric skip autoencodervia an Adversarial Regulariser [6.584748347223698]
  We propose a residual autoencoder architecture, regularised by a residual adversarial network, to generate a more realistic fused image. The residual module serves as primary building for the encoder, decoder and adversarial network. We propose an adversarial regulariser network which would perform supervised learning on the fused image and the original visual image.
  arXiv  Detail & Related papers  (2020-05-01T15:31:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.