Frequency-Aware Guidance for Blind Image Restoration via Diffusion Models

• URL: http://arxiv.org/abs/2411.12450v1
• Date: Tue, 19 Nov 2024 12:18:16 GMT
• Title: Frequency-Aware Guidance for Blind Image Restoration via Diffusion Models
• Authors: Jun Xiao, Zihang Lyu, Hao Xie, Cong Zhang, Yakun Ju, Changjian Shui, Kin-Man Lam,
• Abstract summary: Blind image restoration remains a significant challenge in low-level vision tasks. Guided diffusion models have achieved promising results in blind image restoration. We propose a novel frequency-aware guidance loss that can be integrated into various diffusion models in a plug-and-play manner.
• Score: 20.898262207229873
• License:
• Abstract: Blind image restoration remains a significant challenge in low-level vision tasks. Recently, denoising diffusion models have shown remarkable performance in image synthesis. Guided diffusion models, leveraging the potent generative priors of pre-trained models along with a differential guidance loss, have achieved promising results in blind image restoration. However, these models typically consider data consistency solely in the spatial domain, often resulting in distorted image content. In this paper, we propose a novel frequency-aware guidance loss that can be integrated into various diffusion models in a plug-and-play manner. Our proposed guidance loss, based on 2D discrete wavelet transform, simultaneously enforces content consistency in both the spatial and frequency domains. Experimental results demonstrate the effectiveness of our method in three blind restoration tasks: blind image deblurring, imaging through turbulence, and blind restoration for multiple degradations. Notably, our method achieves a significant improvement in PSNR score, with a remarkable enhancement of 3.72\,dB in image deblurring. Moreover, our method exhibits superior capability in generating images with rich details and reduced distortion, leading to the best visual quality.

Related papers

• FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Process [120.91393949012014]
  FreeEnhance is a framework for content-consistent image enhancement using off-the-shelf image diffusion models. In the noising stage, FreeEnhance is devised to add lighter noise to the region with higher frequency to preserve the high-frequent patterns in the original image. In the denoising stage, we present three target properties as constraints to regularize the predicted noise, enhancing images with high acutance and high visual quality.
  arXiv  Detail & Related papers  (2024-09-11T17:58:50Z)
• DiffLoss: unleashing diffusion model as constraint for training image restoration network [4.8677910801584385]
  We introduce a new perspective that implicitly leverages the diffusion model to assist the training of image restoration network, called DiffLoss. By combining these two designs, the overall loss function is able to improve the perceptual quality of image restoration, resulting in visually pleasing and semantically enhanced outcomes.
  arXiv  Detail & Related papers  (2024-06-27T09:33:24Z)
• Diffusion Reconstruction of Ultrasound Images with Informative Uncertainty [5.375425938215277]
  Enhancing ultrasound image quality involves balancing concurrent factors like contrast, resolution, and speckle preservation. We propose a hybrid approach leveraging advances in diffusion models. We conduct comprehensive experiments on simulated, in-vitro, and in-vivo data, demonstrating the efficacy of our approach.
  arXiv  Detail & Related papers  (2023-10-31T16:51:40Z)
• Reconstruct-and-Generate Diffusion Model for Detail-Preserving Image Denoising [16.43285056788183]
  We propose a novel approach called the Reconstruct-and-Generate Diffusion Model (RnG) Our method leverages a reconstructive denoising network to recover the majority of the underlying clean signal. It employs a diffusion algorithm to generate residual high-frequency details, thereby enhancing visual quality.
  arXiv  Detail & Related papers  (2023-09-19T16:01:20Z)
• Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey [96.99328714941657]
  We present a comprehensive review of recent diffusion model-based methods on image restoration. We classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR. We propose five potential and challenging directions for the future research of diffusion model-based IR.
  arXiv  Detail & Related papers  (2023-08-18T08:40:38Z)
• Learning A Coarse-to-Fine Diffusion Transformer for Image Restoration [39.071637725773314]
  We propose a coarse-to-fine diffusion Transformer (C2F-DFT) for image restoration. C2F-DFT contains diffusion self-attention (DFSA) and diffusion feed-forward network (DFN) In the coarse training stage, our C2F-DFT estimates noises and then generates the final clean image by a sampling algorithm.
  arXiv  Detail & Related papers  (2023-08-17T01:59:59Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction [60.52716381465063]
  We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality pseudo-observations. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times.
  arXiv  Detail & Related papers  (2023-05-24T14:00:32Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal [74.86415440438051]
  We propose a unified diffusion framework that integrates both the image and degradation priors for highly effective shadow removal. Our model achieves a significant improvement in terms of PSNR, increasing from 31.69dB to 34.73dB over SRD dataset.
  arXiv  Detail & Related papers  (2022-12-09T07:48:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.