Multiscale Structure Guided Diffusion for Image Deblurring
- URL: http://arxiv.org/abs/2212.01789v3
- Date: Tue, 12 Dec 2023 19:15:59 GMT
- Title: Multiscale Structure Guided Diffusion for Image Deblurring
- Authors: Mengwei Ren, Mauricio Delbracio, Hossein Talebi, Guido Gerig, Peyman
Milanfar
- Abstract summary: Diffusion Probabilistic Models (DPMs) have been employed for image deblurring.
We introduce a simple yet effective multiscale structure guidance as an implicit bias.
We demonstrate more robust deblurring results with fewer artifacts on unseen data.
- Score: 24.09642909404091
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Diffusion Probabilistic Models (DPMs) have recently been employed for image
deblurring, formulated as an image-conditioned generation process that maps
Gaussian noise to the high-quality image, conditioned on the blurry input.
Image-conditioned DPMs (icDPMs) have shown more realistic results than
regression-based methods when trained on pairwise in-domain data. However,
their robustness in restoring images is unclear when presented with
out-of-domain images as they do not impose specific degradation models or
intermediate constraints. To this end, we introduce a simple yet effective
multiscale structure guidance as an implicit bias that informs the icDPM about
the coarse structure of the sharp image at the intermediate layers. This guided
formulation leads to a significant improvement of the deblurring results,
particularly on unseen domain. The guidance is extracted from the latent space
of a regression network trained to predict the clean-sharp target at multiple
lower resolutions, thus maintaining the most salient sharp structures. With
both the blurry input and multiscale guidance, the icDPM model can better
understand the blur and recover the clean image. We evaluate a single-dataset
trained model on diverse datasets and demonstrate more robust deblurring
results with fewer artifacts on unseen data. Our method outperforms existing
baselines, achieving state-of-the-art perceptual quality while keeping
competitive distortion metrics.
Related papers
- DP-MDM: Detail-Preserving MR Reconstruction via Multiple Diffusion Models [7.601874398726257]
We propose a comprehensive detail-preserving reconstruction method using multiple diffusion models.
The framework effective-ly represents multi-scale sampled data, taking into ac-count the sparsity of the inverted pyramid architecture.
The proposed method was evaluated by con-ducting experiments on clinical and public datasets.
arXiv Detail & Related papers (2024-05-09T13:37:18Z) - Mitigating Data Consistency Induced Discrepancy in Cascaded Diffusion Models for Sparse-view CT Reconstruction [4.227116189483428]
This study introduces a novel Cascaded Diffusion with Discrepancy Mitigation framework.
It includes the low-quality image generation in latent space and the high-quality image generation in pixel space.
It minimizes computational costs by moving some inference steps from pixel space to latent space.
arXiv Detail & Related papers (2024-03-14T12:58:28Z) - Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation.
We present GradPaint, which steers the generation towards a globally coherent image.
We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
arXiv Detail & Related papers (2023-09-18T09:36:24Z) - ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model.
Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models.
The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
arXiv Detail & Related papers (2023-07-15T04:48:35Z) - Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance.
We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring.
Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T12:18:20Z) - Generative Diffusion Prior for Unified Image Restoration and Enhancement [62.76390152617949]
Existing image restoration methods mostly leverage the posterior distribution of natural images.
We propose the Generative Diffusion Prior (GDP) to effectively model the posterior distributions in an unsupervised sampling manner.
GDP utilizes a pre-train denoising diffusion generative model (DDPM) for solving linear inverse, non-linear, or blind problems.
arXiv Detail & Related papers (2023-04-03T16:52:43Z) - Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness.
We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
arXiv Detail & Related papers (2023-03-06T11:51:28Z) - Deep Learning-Based Defect Classification and Detection in SEM Images [1.9206693386750882]
In particular, we train RetinaNet models using different ResNet, VGGNet architectures as backbone.
We propose a preference-based ensemble strategy to combine the output predictions from different models in order to achieve better performance on classification and detection of defects.
arXiv Detail & Related papers (2022-06-20T16:34:11Z) - Denoising Diffusion Restoration Models [110.1244240726802]
Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method.
We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
arXiv Detail & Related papers (2022-01-27T20:19:07Z) - Score-based diffusion models for accelerated MRI [35.3148116010546]
We introduce a way to sample data from a conditional distribution given the measurements, such that the model can be readily used for solving inverse problems in imaging.
Our model requires magnitude images only for training, and yet is able to reconstruct complex-valued data, and even extends to parallel imaging.
arXiv Detail & Related papers (2021-10-08T08:42:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.