Latent Diffusion Prior Enhanced Deep Unfolding for Spectral Image
Reconstruction
- URL: http://arxiv.org/abs/2311.14280v1
- Date: Fri, 24 Nov 2023 04:55:20 GMT
- Title: Latent Diffusion Prior Enhanced Deep Unfolding for Spectral Image
Reconstruction
- Authors: Zongliang Wu, Ruiying Lu, Ying Fu and Xin Yuan
- Abstract summary: Snapshot spectral imaging reconstruction aims to reconstruct three-dimensional spatial-spectral images from a single-shot two-dimensional compressed measurement.
We introduce a generative model, namely the latent diffusion model (LDM), to generate degradation-free prior to deep unfolding method.
- Score: 19.1301471218022
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Snapshot compressive spectral imaging reconstruction aims to reconstruct
three-dimensional spatial-spectral images from a single-shot two-dimensional
compressed measurement. Existing state-of-the-art methods are mostly based on
deep unfolding structures but have intrinsic performance bottlenecks: $i$) the
ill-posed problem of dealing with heavily degraded measurement, and $ii$) the
regression loss-based reconstruction models being prone to recover images with
few details. In this paper, we introduce a generative model, namely the latent
diffusion model (LDM), to generate degradation-free prior to enhance the
regression-based deep unfolding method. Furthermore, to overcome the large
computational cost challenge in LDM, we propose a lightweight model to generate
knowledge priors in deep unfolding denoiser, and integrate these priors to
guide the reconstruction process for compensating high-quality spectral signal
details. Numeric and visual comparisons on synthetic and real-world datasets
illustrate the superiority of our proposed method in both reconstruction
quality and computational efficiency. Code will be released.
Related papers
- Reti-Diff: Illumination Degradation Image Restoration with Retinex-based
Latent Diffusion Model [59.08821399652483]
Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination.
Among these algorithms, diffusion model (DM)-based methods have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution.
We propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task.
Reti-Diff comprises two key components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RG
arXiv Detail & Related papers (2023-11-20T09:55:06Z) - DiffSCI: Zero-Shot Snapshot Compressive Imaging via Iterative Spectral
Diffusion Model [18.25548360119976]
This paper endeavors to advance the precision of snapshot compressive imaging (SCI) reconstruction for multispectral image (MSI)
We propose a novel structured zero-shot diffusion model, dubbed DiffSCI.
We present extensive testing to show that DiffSCI exhibits discernible performance enhancements over prevailing self-supervised and zero-shot approaches.
arXiv Detail & Related papers (2023-11-19T20:27:14Z) - Reconstruct-and-Generate Diffusion Model for Detail-Preserving Image
Denoising [16.43285056788183]
We propose a novel approach called the Reconstruct-and-Generate Diffusion Model (RnG)
Our method leverages a reconstructive denoising network to recover the majority of the underlying clean signal.
It employs a diffusion algorithm to generate residual high-frequency details, thereby enhancing visual quality.
arXiv Detail & Related papers (2023-09-19T16:01:20Z) - Reconstruction Distortion of Learned Image Compression with
Imperceptible Perturbations [69.25683256447044]
We introduce an attack approach designed to effectively degrade the reconstruction quality of Learned Image Compression (LIC)
We generate adversarial examples by introducing a Frobenius norm-based loss function to maximize the discrepancy between original images and reconstructed adversarial examples.
Experiments conducted on the Kodak dataset using various LIC models demonstrate effectiveness.
arXiv Detail & Related papers (2023-06-01T20:21:05Z) - Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance.
We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring.
Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T12:18:20Z) - A Fast Alternating Minimization Algorithm for Coded Aperture Snapshot
Spectral Imaging Based on Sparsity and Deep Image Priors [8.890754092562918]
Coded aperture snapshot spectral imaging (CASSI) is a technique used to reconstruct three-dimensional hyperspectral images (HSIs)
This paper proposes a fast alternating minimization algorithm based on the sparsity and deep image priors (Fama-P) of natural images.
arXiv Detail & Related papers (2022-06-12T03:29:14Z) - Multi-Channel Convolutional Analysis Operator Learning for Dual-Energy
CT Reconstruction [108.06731611196291]
We develop a multi-channel convolutional analysis operator learning (MCAOL) method to exploit common spatial features within attenuation images at different energies.
We propose an optimization method which jointly reconstructs the attenuation images at low and high energies with a mixed norm regularization on the sparse features.
arXiv Detail & Related papers (2022-03-10T14:22:54Z) - Spectral Compressive Imaging Reconstruction Using Convolution and
Contextual Transformer [6.929652454131988]
We propose a hybrid network module, namely CCoT (Contextual Transformer) block, which can acquire the inductive bias ability of transformer simultaneously.
We integrate the proposed CCoT block into deep unfolding framework based on the generalized alternating projection algorithm, and further propose the GAP-CT network.
arXiv Detail & Related papers (2022-01-15T06:30:03Z) - Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
Blind image restoration is a common yet challenging problem in computer vision.
We propose a novel blind image restoration method, aiming to integrate both the advantages of them.
Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
arXiv Detail & Related papers (2020-08-25T03:30:53Z) - Attention Based Real Image Restoration [48.933507352496726]
Deep convolutional neural networks perform better on images containing synthetic degradations.
This paper proposes a novel single-stage blind real image restoration network (R$2$Net)
arXiv Detail & Related papers (2020-04-26T04:21:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.