DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration

• URL: http://arxiv.org/abs/2303.06682v2
• Date: Sun, 19 Mar 2023 10:43:06 GMT
• Title: DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration
• Authors: Yuchun Miao and Lefei Zhang and Liangpei Zhang and Dacheng Tao
• Abstract summary: Self-supervised diffusion model for hyperspectral image restoration is proposed. textttDDS2M enjoys stronger ability to generalization compared to existing diffusion-based methods. Experiments on HSI denoising, noisy HSI completion and super-resolution on a variety of HSIs demonstrate textttDDS2M's superiority over the existing task-specific state-of-the-arts.
• Score: 103.79030498369319
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have recently received a surge of interest due to their impressive performance for image restoration, especially in terms of noise robustness. However, existing diffusion-based methods are trained on a large amount of training data and perform very well in-distribution, but can be quite susceptible to distribution shift. This is especially inappropriate for data-starved hyperspectral image (HSI) restoration. To tackle this problem, this work puts forth a self-supervised diffusion model for HSI restoration, namely Denoising Diffusion Spatio-Spectral Model (\texttt{DDS2M}), which works by inferring the parameters of the proposed Variational Spatio-Spectral Module (VS2M) during the reverse diffusion process, solely using the degraded HSI without any extra training data. In VS2M, a variational inference-based loss function is customized to enable the untrained spatial and spectral networks to learn the posterior distribution, which serves as the transitions of the sampling chain to help reverse the diffusion process. Benefiting from its self-supervised nature and the diffusion process, \texttt{DDS2M} enjoys stronger generalization ability to various HSIs compared to existing diffusion-based methods and superior robustness to noise compared to existing HSI restoration methods. Extensive experiments on HSI denoising, noisy HSI completion and super-resolution on a variety of HSIs demonstrate \texttt{DDS2M}'s superiority over the existing task-specific state-of-the-arts.

Related papers

• Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
  This study proposes a novel and efficient diffusion model for image restoration. Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration. Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
  arXiv  Detail & Related papers  (2024-03-12T05:06:07Z)
• HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models [38.74983301496911]
  Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations. Existing model-based methods have limitations in accurately modeling the complex image characteristics. This paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff)
  arXiv  Detail & Related papers  (2024-02-24T17:15:05Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Residual Denoising Diffusion Models [12.698791701225499]
  We propose a novel dual diffusion process that decouples the traditional single denoising diffusion process into residual diffusion and noise diffusion. This dual diffusion framework expands the denoising-based diffusion models into a unified and interpretable model for both image generation and restoration. We provide code and pre-trained models to encourage further exploration, application, and development of our innovative framework.
  arXiv  Detail & Related papers  (2023-08-25T23:54:15Z)
• Learning A Coarse-to-Fine Diffusion Transformer for Image Restoration [39.071637725773314]
  We propose a coarse-to-fine diffusion Transformer (C2F-DFT) for image restoration. C2F-DFT contains diffusion self-attention (DFSA) and diffusion feed-forward network (DFN) In the coarse training stage, our C2F-DFT estimates noises and then generates the final clean image by a sampling algorithm.
  arXiv  Detail & Related papers  (2023-08-17T01:59:59Z)
• SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translation [96.11061713135385]
  This work presents a new score-decomposed diffusion model to explicitly optimize the tangled distributions during image generation. We equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold. SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.
  arXiv  Detail & Related papers  (2023-08-04T06:21:57Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• DiffUCD:Unsupervised Hyperspectral Image Change Detection with Semantic Correlation Diffusion Model [46.68717345017946]
  Hyperspectral image change detection (HSI-CD) has emerged as a crucial research area in remote sensing. We propose a novel unsupervised HSI-CD with semantic correlation diffusion model (DiffUCD) Our method can achieve comparable results to those fully supervised methods requiring numerous samples.
  arXiv  Detail & Related papers  (2023-05-21T09:21:41Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• High-Frequency Space Diffusion Models for Accelerated MRI [7.985113617260289]
  Diffusion models with continuous differential equations (SDEs) have shown superior performances in image generation. We propose a novel SDE tailored specifically for magnetic resonance (MR) reconstruction with the diffusion process in high-frequency space. This approach ensures determinism in the fully sampled low-frequency regions and accelerates the sampling procedure of reverse diffusion.
  arXiv  Detail & Related papers  (2022-08-10T14:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.