End-to-End Learning for Simultaneously Generating Decision Map and Multi-Focus Image Fusion Result

• URL: http://arxiv.org/abs/2010.08751v3
• Date: Tue, 23 Mar 2021 07:34:42 GMT
• Title: End-to-End Learning for Simultaneously Generating Decision Map and Multi-Focus Image Fusion Result
• Authors: Boyuan Ma, Xiang Yin, Di Wu, Xiaojuan Ban
• Abstract summary: The aim of multi-focus image fusion is to gather focused regions of different images to generate a unique all-in-focus fused image. Most of the existing deep learning structures failed to balance fusion quality and end-to-end implementation convenience. We propose a cascade network to simultaneously generate decision map and fused result with an end-to-end training procedure.
• Score: 7.564462759345851
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The general aim of multi-focus image fusion is to gather focused regions of different images to generate a unique all-in-focus fused image. Deep learning based methods become the mainstream of image fusion by virtue of its powerful feature representation ability. However, most of the existing deep learning structures failed to balance fusion quality and end-to-end implementation convenience. End-to-end decoder design often leads to unrealistic result because of its non-linear mapping mechanism. On the other hand, generating an intermediate decision map achieves better quality for the fused image, but relies on the rectification with empirical post-processing parameter choices. In this work, to handle the requirements of both output image quality and comprehensive simplicity of structure implementation, we propose a cascade network to simultaneously generate decision map and fused result with an end-to-end training procedure. It avoids the dependence on empirical post-processing methods in the inference stage. To improve the fusion quality, we introduce a gradient aware loss function to preserve gradient information in output fused image. In addition, we design a decision calibration strategy to decrease the time consumption in the application of multiple images fusion. Extensive experiments are conducted to compare with 19 different state-of-the-art multi-focus image fusion structures with 6 assessment metrics. The results prove that our designed structure can generally ameliorate the output fused image quality, while implementation efficiency increases over 30\% for multiple images fusion.

Related papers

• Fusion from Decomposition: A Self-Supervised Approach for Image Fusion and Beyond [74.96466744512992]
  The essence of image fusion is to integrate complementary information from source images. DeFusion++ produces versatile fused representations that can enhance the quality of image fusion and the effectiveness of downstream high-level vision tasks.
  arXiv  Detail & Related papers  (2024-10-16T06:28:49Z)
• Generation and Recombination for Multifocus Image Fusion with Free Number of Inputs [17.32596568119519]
  Multifocus image fusion is an effective way to overcome the limitation of optical lenses. Previous methods assume that the focused areas of the two source images are complementary, making it impossible to achieve simultaneous fusion of multiple images. In GRFusion, focus property detection of each source image can be implemented independently, enabling simultaneous fusion of multiple source images.
  arXiv  Detail & Related papers  (2023-09-09T01:47:56Z)
• A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion [69.10255211811007]
  We present a Task-guided, Implicit-searched and Meta- generalizationd (TIM) deep model to address the image fusion problem in a challenging real-world scenario. Specifically, we propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion. Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency.
  arXiv  Detail & Related papers  (2023-05-25T08:54:08Z)
• Searching a Compact Architecture for Robust Multi-Exposure Image Fusion [55.37210629454589]
  Two major stumbling blocks hinder the development, including pixel misalignment and inefficient inference. This study introduces an architecture search-based paradigm incorporating self-alignment and detail repletion modules for robust multi-exposure image fusion. The proposed method outperforms various competitive schemes, achieving a noteworthy 3.19% improvement in PSNR for general scenarios and an impressive 23.5% enhancement in misaligned scenarios.
  arXiv  Detail & Related papers  (2023-05-20T17:01:52Z)
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion [72.8898811120795]
  We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion. Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
  arXiv  Detail & Related papers  (2022-11-20T12:02:07Z)
• Unsupervised Image Fusion Method based on Feature Mutual Mapping [16.64607158983448]
  We propose an unsupervised adaptive image fusion method to address the above issues. We construct a global map to measure the connections of pixels between the input source images. Our method achieves superior performance in both visual perception and objective evaluation.
  arXiv  Detail & Related papers  (2022-01-25T07:50:14Z)
• TransFuse: A Unified Transformer-based Image Fusion Framework using Self-supervised Learning [5.849513679510834]
  Image fusion is a technique to integrate information from multiple source images with complementary information to improve the richness of a single image. Two-stage methods avoid the need of large amount of task-specific training data by training encoder-decoder network on large natural image datasets. We propose a destruction-reconstruction based self-supervised training scheme to encourage the network to learn task-specific features.
  arXiv  Detail & Related papers  (2022-01-19T07:30:44Z)
• UFA-FUSE: A novel deep supervised and hybrid model for multi-focus image fusion [4.105749631623888]
  Traditional and deep learning-based fusion methods generate the intermediate decision map through a series of post-processing procedures. Inspired by the image reconstruction techniques based on deep learning, we propose a multi-focus image fusion network framework. We show that the proposed approach for multi-focus image fusion achieves remarkable fusion performance compared to 19 state-of-the-art fusion methods.
  arXiv  Detail & Related papers  (2021-01-12T14:33:13Z)
• Deep Image Compositing [93.75358242750752]
  We propose a new method which can automatically generate high-quality image composites without any user input. Inspired by Laplacian pyramid blending, a dense-connected multi-stream fusion network is proposed to effectively fuse the information from the foreground and background images. Experiments show that the proposed method can automatically generate high-quality composites and outperforms existing methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-11-04T06:12:24Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.