Deep Convolutional Sparse Coding Networks for Image Fusion
- URL: http://arxiv.org/abs/2005.08448v1
- Date: Mon, 18 May 2020 04:12:01 GMT
- Title: Deep Convolutional Sparse Coding Networks for Image Fusion
- Authors: Shuang Xu, Zixiang Zhao, Yicheng Wang, Chunxia Zhang, Junmin Liu,
Jiangshe Zhang
- Abstract summary: Deep learning has emerged as an important tool for image fusion.
This paper presents three deep convolutional sparse coding (CSC) networks for three kinds of image fusion tasks.
- Score: 29.405149234582623
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image fusion is a significant problem in many fields including digital
photography, computational imaging and remote sensing, to name but a few.
Recently, deep learning has emerged as an important tool for image fusion. This
paper presents three deep convolutional sparse coding (CSC) networks for three
kinds of image fusion tasks (i.e., infrared and visible image fusion,
multi-exposure image fusion, and multi-modal image fusion). The CSC model and
the iterative shrinkage and thresholding algorithm are generalized into
dictionary convolution units. As a result, all hyper-parameters are learned
from data. Our extensive experiments and comprehensive comparisons reveal the
superiority of the proposed networks with regard to quantitative evaluation and
visual inspection.
Related papers
- Fusion from Decomposition: A Self-Supervised Approach for Image Fusion and Beyond [74.96466744512992]
The essence of image fusion is to integrate complementary information from source images.
DeFusion++ produces versatile fused representations that can enhance the quality of image fusion and the effectiveness of downstream high-level vision tasks.
arXiv Detail & Related papers (2024-10-16T06:28:49Z) - A Task-guided, Implicitly-searched and Meta-initialized Deep Model for
Image Fusion [69.10255211811007]
We present a Task-guided, Implicit-searched and Meta- generalizationd (TIM) deep model to address the image fusion problem in a challenging real-world scenario.
Specifically, we propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion.
Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency.
arXiv Detail & Related papers (2023-05-25T08:54:08Z) - An Interactively Reinforced Paradigm for Joint Infrared-Visible Image
Fusion and Saliency Object Detection [59.02821429555375]
This research focuses on the discovery and localization of hidden objects in the wild and serves unmanned systems.
Through empirical analysis, infrared and visible image fusion (IVIF) enables hard-to-find objects apparent.
multimodal salient object detection (SOD) accurately delineates the precise spatial location of objects within the picture.
arXiv Detail & Related papers (2023-05-17T06:48:35Z) - CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature
Ensemble for Multi-modality Image Fusion [72.8898811120795]
We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion.
Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
arXiv Detail & Related papers (2022-11-20T12:02:07Z) - Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network.
We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
arXiv Detail & Related papers (2022-04-19T17:59:45Z) - TransFuse: A Unified Transformer-based Image Fusion Framework using
Self-supervised Learning [5.849513679510834]
Image fusion is a technique to integrate information from multiple source images with complementary information to improve the richness of a single image.
Two-stage methods avoid the need of large amount of task-specific training data by training encoder-decoder network on large natural image datasets.
We propose a destruction-reconstruction based self-supervised training scheme to encourage the network to learn task-specific features.
arXiv Detail & Related papers (2022-01-19T07:30:44Z) - Image deblurring based on lightweight multi-information fusion network [6.848061582669787]
We propose a lightweight multiinformation fusion network (LMFN) for image deblurring.
In the encoding stage, the image feature is reduced to various smallscale spaces for multi-scale information extraction and fusion.
Then, a distillation network is used in the decoding stage, which allows the network benefit the most from residual learning.
Our network can achieve state-of-the-art image deblurring result with smaller number of parameters and outperforms existing methods in model complexity.
arXiv Detail & Related papers (2021-01-14T00:37:37Z) - FuseVis: Interpreting neural networks for image fusion using per-pixel
saliency visualization [10.156766309614113]
Unsupervised learning based convolutional neural networks (CNNs) have been utilized for different types of image fusion tasks.
It is challenging to analyze the reliability of these CNNs for the image fusion tasks since no groundtruth is available.
We present a novel real-time visualization tool, named FuseVis, with which the end-user can compute per-pixel saliency maps.
arXiv Detail & Related papers (2020-12-06T10:03:02Z) - Real-MFF: A Large Realistic Multi-focus Image Dataset with Ground Truth [58.226535803985804]
We introduce a large and realistic multi-focus dataset called Real-MFF.
The dataset contains 710 pairs of source images with corresponding ground truth images.
We evaluate 10 typical multi-focus algorithms on this dataset for the purpose of illustration.
arXiv Detail & Related papers (2020-03-28T12:33:46Z) - 3D Gated Recurrent Fusion for Semantic Scene Completion [32.86736222106503]
This paper tackles the problem of data fusion in the semantic scene completion (SSC) task.
We propose a 3D gated recurrent fusion network (GRFNet), which learns to adaptively select and fuse the relevant information from depth and RGB.
Experiments on two benchmark datasets demonstrate the superior performance and the effectiveness of the proposed GRFNet for data fusion in SSC.
arXiv Detail & Related papers (2020-02-17T21:45:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.