Image fusion using symmetric skip autoencodervia an Adversarial Regulariser

• URL: http://arxiv.org/abs/2005.00447v2
• Date: Thu, 4 Jun 2020 07:33:25 GMT
• Title: Image fusion using symmetric skip autoencodervia an Adversarial Regulariser
• Authors: Snigdha Bhagat, S. D. Joshi, Brejesh Lall
• Abstract summary: We propose a residual autoencoder architecture, regularised by a residual adversarial network, to generate a more realistic fused image. The residual module serves as primary building for the encoder, decoder and adversarial network. We propose an adversarial regulariser network which would perform supervised learning on the fused image and the original visual image.
• Score: 6.584748347223698
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is a challenging task to extract the best of both worlds by combining the spatial characteristics of a visible image and the spectral content of an infrared image. In this work, we propose a spatially constrained adversarial autoencoder that extracts deep features from the infrared and visible images to obtain a more exhaustive and global representation. In this paper, we propose a residual autoencoder architecture, regularised by a residual adversarial network, to generate a more realistic fused image. The residual module serves as primary building for the encoder, decoder and adversarial network, as an add on the symmetric skip connections perform the functionality of embedding the spatial characteristics directly from the initial layers of encoder structure to the decoder part of the network. The spectral information in the infrared image is incorporated by adding the feature maps over several layers in the encoder part of the fusion structure, which makes inference on both the visual and infrared images separately. In order to efficiently optimize the parameters of the network, we propose an adversarial regulariser network which would perform supervised learning on the fused image and the original visual image.

Related papers

• Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution [54.293362972473595]
  Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Current approaches to address SR tasks are either dedicated to extracting RGB image features or assuming similar degradation patterns. We propose a Contourlet refinement gate framework to restore infrared modal-specific features while preserving spectral distribution fidelity.
  arXiv  Detail & Related papers  (2024-11-19T14:24:03Z)
• MaeFuse: Transferring Omni Features with Pretrained Masked Autoencoders for Infrared and Visible Image Fusion via Guided Training [57.18758272617101]
  MaeFuse is a novel autoencoder model designed for infrared and visible image fusion (IVIF) Our model utilizes a pretrained encoder from Masked Autoencoders (MAE), which facilities the omni features extraction for low-level reconstruction and high-level vision tasks. MaeFuse not only introduces a novel perspective in the realm of fusion techniques but also stands out with impressive performance across various public datasets.
  arXiv  Detail & Related papers  (2024-04-17T02:47:39Z)
• RemoteNet: Remote Sensing Image Segmentation Network based on Global-Local Information [7.953644697658355]
  We propose a remote sensing image segmentation network, RemoteNet, for semantic segmentation of remote sensing images. We capture the global and local features by leveraging the benefits of the transformer and convolution mechanisms.
  arXiv  Detail & Related papers  (2023-02-25T14:09:52Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion [72.8898811120795]
  We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion. Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
  arXiv  Detail & Related papers  (2022-11-20T12:02:07Z)
• SoftPool++: An Encoder-Decoder Network for Point Cloud Completion [93.54286830844134]
  We propose a novel convolutional operator for the task of point cloud completion. The proposed operator does not require any max-pooling or voxelization operation. We show that our approach achieves state-of-the-art performance in shape completion at low and high resolutions.
  arXiv  Detail & Related papers  (2022-05-08T15:31:36Z)
• A Joint Convolution Auto-encoder Network for Infrared and Visible Image Fusion [7.799758067671958]
  We design a joint convolution auto-encoder (JCAE) network for infrared and visible image fusion. Inspired by the infrared cognition ability of crotalinae animals, we design a joint convolution auto-encoder (JCAE) network for infrared and visible image fusion.
  arXiv  Detail & Related papers  (2022-01-26T03:49:27Z)
• Convolutional Autoencoder for Blind Hyperspectral Image Unmixing [0.0]
  spectral unmixing is a technique to decompose a mixed pixel into two fundamental representatives: endmembers and abundances. In this paper, a novel architecture is proposed to perform blind unmixing on hyperspectral images.
  arXiv  Detail & Related papers  (2020-11-18T17:41:31Z)
• When Image Decomposition Meets Deep Learning: A Novel Infrared and Visible Image Fusion Method [27.507158159317417]
  Infrared and visible image fusion is a hot topic in image processing and image enhancement. We propose a novel dual-stream auto-encoder based fusion network.
  arXiv  Detail & Related papers  (2020-09-02T19:32:28Z)
• Guided Deep Decoder: Unsupervised Image Pair Fusion [26.999346037307888]
  The proposed network is composed of an encoder-decoder network that exploits multi-scale features of a guidance image and a deep decoder network that generates an output image. Our results show that the proposed network can achieve state-of-the-art performance in various image fusion problems.
  arXiv  Detail & Related papers  (2020-07-23T03:06:06Z)
• Beyond Single Stage Encoder-Decoder Networks: Deep Decoders for Semantic Image Segmentation [56.44853893149365]
  Single encoder-decoder methodologies for semantic segmentation are reaching their peak in terms of segmentation quality and efficiency per number of layers. We propose a new architecture based on a decoder which uses a set of shallow networks for capturing more information content. In order to further improve the architecture we introduce a weight function which aims to re-balance classes to increase the attention of the networks to under-represented objects.
  arXiv  Detail & Related papers  (2020-07-19T18:44:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.