Physics Driven Deep Retinex Fusion for Adaptive Infrared and Visible
Image Fusion
- URL: http://arxiv.org/abs/2112.02869v4
- Date: Tue, 21 Mar 2023 06:34:09 GMT
- Title: Physics Driven Deep Retinex Fusion for Adaptive Infrared and Visible
Image Fusion
- Authors: Yuanjie Gu, Zhibo Xiao, Yinghan Guan, Haoran Dai, Cheng Liu, Liang Xue
and Shouyu Wang
- Abstract summary: Convolutional neural networks have turned into an illustrious tool for image fusion and super-resolution.
Deep Retinex Fusion (DRF) is a dataset-free method for adaptive infrared (IR) and visible (VIS) image super-resolution fusion.
DRF can adaptively balance IR and VIS information and has good noise immunity.
- Score: 2.9591172349593102
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Convolutional neural networks have turned into an illustrious tool for image
fusion and super-resolution. However, their excellent performance cannot work
without large fixed-paired datasets; and additionally, these high-demanded
ground truth data always cannot be obtained easily in fusion tasks. In this
study, we show that, the structures of generative networks capture a great deal
of image feature priors, and then these priors are sufficient to reconstruct
high-quality fused super-resolution result using only low-resolution inputs. By
this way, we propose a novel self-supervised dataset-free method for adaptive
infrared (IR) and visible (VIS) image super-resolution fusion named Deep
Retinex Fusion (DRF). The key idea of DRF is first generating component priors
which are disentangled from physical model using our designed generative
networks ZipperNet, LightingNet and AdjustingNet, then combining these priors
which captured by networks via adaptive fusion loss functions based on Retinex
theory, and finally reconstructing the super-resolution fusion results.
Furthermore, in order to verify the effectiveness of our reported DRF, both
qualitative and quantitative experiments via comparing with other
state-of-the-art methods are performed using different test sets. These results
prove that, comparing with large datasets trained methods, DRF which works
without any dataset achieves the best super-resolution fusion performance; and
more importantly, DRF can adaptively balance IR and VIS information and has
good noise immunity. DRF codes are open source available at
https://github.com/GuYuanjie/Deep-Retinex-fusion.
Related papers
- NeRF-DetS: Enhancing Multi-View 3D Object Detection with Sampling-adaptive Network of Continuous NeRF-based Representation [60.47114985993196]
NeRF-Det unifies the tasks of novel view arithmetic and 3D perception.
We introduce a novel 3D perception network structure, NeRF-DetS.
NeRF-DetS outperforms competitive NeRF-Det on the ScanNetV2 dataset.
arXiv Detail & Related papers (2024-04-22T06:59:03Z) - SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised
Learning for Robust Infrared Small Target Detection [53.19618419772467]
Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds.
With the development of Transformer, the scale of SIRST models is constantly increasing.
With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
arXiv Detail & Related papers (2024-03-08T16:14:54Z) - Tensor Factorization for Leveraging Cross-Modal Knowledge in
Data-Constrained Infrared Object Detection [22.60228799622782]
Key bottleneck in object detection in IR images is lack of sufficient labeled training data.
We seek to leverage cues from the RGB modality to scale object detectors to the IR modality, while preserving model performance in the RGB modality.
We first pretrain these factor matrices on the RGB modality, for which plenty of training data are assumed to exist and then augment only a few trainable parameters for training on the IR modality to avoid over-fitting.
arXiv Detail & Related papers (2023-09-28T16:55:52Z) - RBSR: Efficient and Flexible Recurrent Network for Burst
Super-Resolution [57.98314517861539]
Burst super-resolution (BurstSR) aims at reconstructing a high-resolution (HR) image from a sequence of low-resolution (LR) and noisy images.
In this paper, we suggest fusing cues frame-by-frame with an efficient and flexible recurrent network.
arXiv Detail & Related papers (2023-06-30T12:14:13Z) - Denoising Diffusion Models for Plug-and-Play Image Restoration [135.6359475784627]
This paper proposes DiffPIR, which integrates the traditional plug-and-play method into the diffusion sampling framework.
Compared to plug-and-play IR methods that rely on discriminative Gaussian denoisers, DiffPIR is expected to inherit the generative ability of diffusion models.
arXiv Detail & Related papers (2023-05-15T20:24:38Z) - CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for
Multi-Modality Image Fusion [138.40422469153145]
We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network.
We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
arXiv Detail & Related papers (2022-11-26T02:40:28Z) - Pyramid Grafting Network for One-Stage High Resolution Saliency
Detection [29.013012579688347]
We propose a one-stage framework called Pyramid Grafting Network (PGNet) to extract features from different resolution images independently.
An attention-based Cross-Model Grafting Module (CMGM) is proposed to enable CNN branch to combine broken detailed information more holistically.
We contribute a new Ultra-High-Resolution Saliency Detection dataset UHRSD, containing 5,920 images at 4K-8K resolutions.
arXiv Detail & Related papers (2022-04-11T12:22:21Z) - RFN-Nest: An end-to-end residual fusion network for infrared and visible
images [37.935940961760785]
We propose an end-to-end fusion network architecture (RFN-Nest) for infrared and visible image fusion.
A novel detail-preserving loss function, and a feature enhancing loss function are proposed to train RFN.
The experimental results on public domain data sets show that, compared with the existing methods, our end-to-end fusion network delivers a better performance than the state-of-the-art methods.
arXiv Detail & Related papers (2021-03-07T07:29:50Z) - Efficient and Model-Based Infrared and Visible Image Fusion Via
Algorithm Unrolling [24.83209572888164]
Infrared and visible image fusion (IVIF) expects to obtain images that retain thermal radiation information from infrared images and texture details from visible images.
A model-based convolutional neural network (CNN) model is proposed to overcome the shortcomings of traditional CNN-based IVIF models.
arXiv Detail & Related papers (2020-05-12T16:15:56Z) - FD-GAN: Generative Adversarial Networks with Fusion-discriminator for
Single Image Dehazing [48.65974971543703]
We propose a fully end-to-end Generative Adversarial Networks with Fusion-discriminator (FD-GAN) for image dehazing.
Our model can generator more natural and realistic dehazed images with less color distortion and fewer artifacts.
Experiments have shown that our method reaches state-of-the-art performance on both public synthetic datasets and real-world images.
arXiv Detail & Related papers (2020-01-20T04:36:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.