Guided Deep Decoder: Unsupervised Image Pair Fusion

• URL: http://arxiv.org/abs/2007.11766v1
• Date: Thu, 23 Jul 2020 03:06:06 GMT
• Title: Guided Deep Decoder: Unsupervised Image Pair Fusion
• Authors: Tatsumi Uezato, Danfeng Hong, Naoto Yokoya, Wei He
• Abstract summary: The proposed network is composed of an encoder-decoder network that exploits multi-scale features of a guidance image and a deep decoder network that generates an output image. Our results show that the proposed network can achieve state-of-the-art performance in various image fusion problems.
• Score: 26.999346037307888
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The fusion of input and guidance images that have a tradeoff in their information (e.g., hyperspectral and RGB image fusion or pansharpening) can be interpreted as one general problem. However, previous studies applied a task-specific handcrafted prior and did not address the problems with a unified approach. To address this limitation, in this study, we propose a guided deep decoder network as a general prior. The proposed network is composed of an encoder-decoder network that exploits multi-scale features of a guidance image and a deep decoder network that generates an output image. The two networks are connected by feature refinement units to embed the multi-scale features of the guidance image into the deep decoder network. The proposed network allows the network parameters to be optimized in an unsupervised way without training data. Our results show that the proposed network can achieve state-of-the-art performance in various image fusion problems.

Related papers

• PottsMGNet: A Mathematical Explanation of Encoder-Decoder Based Neural Networks [7.668812831777923]
  We study the encoder-decoder-based network architecture from the algorithmic perspective. We use the two-phase Potts model for image segmentation as an example for our explanations. We show that the resulting discrete PottsMGNet is equivalent to an encoder-decoder-based network.
  arXiv  Detail & Related papers  (2023-07-18T07:48:48Z)
• LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images [98.36300655482196]
  We formulate the fusion task mathematically, and establish a connection between its optimal solution and the network architecture that can implement it. In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model. Based on this novel network architecture, an end-to-end lightweight fusion network is constructed to fuse infrared and visible light images.
  arXiv  Detail & Related papers  (2023-04-11T12:11:23Z)
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion [72.8898811120795]
  We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion. Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
  arXiv  Detail & Related papers  (2022-11-20T12:02:07Z)
• Rethinking Performance Gains in Image Dehazing Networks [25.371802581339576]
  We make minimal modifications to popular U-Net to obtain a compact dehazing network. Specifically, we swap out the convolutional blocks in U-Net for residual blocks with the gating mechanism. With a significantly reduced overhead, gUNet is superior to state-of-the-art methods on multiple image dehazing datasets.
  arXiv  Detail & Related papers  (2022-09-23T07:14:48Z)
• Reducing Redundancy in the Bottleneck Representation of the Autoencoders [98.78384185493624]
  Autoencoders are a type of unsupervised neural networks, which can be used to solve various tasks. We propose a scheme to explicitly penalize feature redundancies in the bottleneck representation. We tested our approach across different tasks: dimensionality reduction using three different dataset, image compression using the MNIST dataset, and image denoising using fashion MNIST.
  arXiv  Detail & Related papers  (2022-02-09T18:48:02Z)
• A Dual-branch Network for Infrared and Visible Image Fusion [20.15854042473049]
  We propose a new method based on dense blocks and GANs. We directly insert the input image-visible light image in each layer of the entire network. Our experiments show that the fused images obtained by our approach achieve good score based on multiple evaluation indicators.
  arXiv  Detail & Related papers  (2021-01-24T04:18:32Z)
• Image deblurring based on lightweight multi-information fusion network [6.848061582669787]
  We propose a lightweight multiinformation fusion network (LMFN) for image deblurring. In the encoding stage, the image feature is reduced to various smallscale spaces for multi-scale information extraction and fusion. Then, a distillation network is used in the decoding stage, which allows the network benefit the most from residual learning. Our network can achieve state-of-the-art image deblurring result with smaller number of parameters and outperforms existing methods in model complexity.
  arXiv  Detail & Related papers  (2021-01-14T00:37:37Z)
• D-Unet: A Dual-encoder U-Net for Image Splicing Forgery Detection and Localization [108.8592577019391]
  Image splicing forgery detection is a global binary classification task that distinguishes the tampered and non-tampered regions by image fingerprints. We propose a novel network called dual-encoder U-Net (D-Unet) for image splicing forgery detection, which employs an unfixed encoder and a fixed encoder. In an experimental comparison study of D-Unet and state-of-the-art methods, D-Unet outperformed the other methods in image-level and pixel-level detection.
  arXiv  Detail & Related papers  (2020-12-03T10:54:02Z)
• Beyond Single Stage Encoder-Decoder Networks: Deep Decoders for Semantic Image Segmentation [56.44853893149365]
  Single encoder-decoder methodologies for semantic segmentation are reaching their peak in terms of segmentation quality and efficiency per number of layers. We propose a new architecture based on a decoder which uses a set of shallow networks for capturing more information content. In order to further improve the architecture we introduce a weight function which aims to re-balance classes to increase the attention of the networks to under-represented objects.
  arXiv  Detail & Related papers  (2020-07-19T18:44:34Z)
• Suppress and Balance: A Simple Gated Network for Salient Object Detection [89.88222217065858]
  We propose a simple gated network (GateNet) to solve both issues at once. With the help of multilevel gate units, the valuable context information from the encoder can be optimally transmitted to the decoder. In addition, we adopt the atrous spatial pyramid pooling based on the proposed "Fold" operation (Fold-ASPP) to accurately localize salient objects of various scales.
  arXiv  Detail & Related papers  (2020-07-16T02:00:53Z)
• Image fusion using symmetric skip autoencodervia an Adversarial Regulariser [6.584748347223698]
  We propose a residual autoencoder architecture, regularised by a residual adversarial network, to generate a more realistic fused image. The residual module serves as primary building for the encoder, decoder and adversarial network. We propose an adversarial regulariser network which would perform supervised learning on the fused image and the original visual image.
  arXiv  Detail & Related papers  (2020-05-01T15:31:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.