Harnessing Diffusion-Yielded Score Priors for Image Restoration

• URL: http://arxiv.org/abs/2507.20590v2
• Date: Tue, 29 Jul 2025 23:59:12 GMT
• Title: Harnessing Diffusion-Yielded Score Priors for Image Restoration
• Authors: Xinqi Lin, Fanghua Yu, Jinfan Hu, Zhiyuan You, Wu Shi, Jimmy S. Ren, Jinjin Gu, Chao Dong,
• Abstract summary: Deep image restoration models aim to learn a mapping from degraded image space to natural image space.<n>Three major classes of methods have emerged, including MSE-based, GAN-based, and diffusion-based methods.<n>We propose a novel method, HYPIR, to address these challenges.
• Score: 29.788482710572307
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep image restoration models aim to learn a mapping from degraded image space to natural image space. However, they face several critical challenges: removing degradation, generating realistic details, and ensuring pixel-level consistency. Over time, three major classes of methods have emerged, including MSE-based, GAN-based, and diffusion-based methods. However, they fail to achieve a good balance between restoration quality, fidelity, and speed. We propose a novel method, HYPIR, to address these challenges. Our solution pipeline is straightforward: it involves initializing the image restoration model with a pre-trained diffusion model and then fine-tuning it with adversarial training. This approach does not rely on diffusion loss, iterative sampling, or additional adapters. We theoretically demonstrate that initializing adversarial training from a pre-trained diffusion model positions the initial restoration model very close to the natural image distribution. Consequently, this initialization improves numerical stability, avoids mode collapse, and substantially accelerates the convergence of adversarial training. Moreover, HYPIR inherits the capabilities of diffusion models with rich user control, enabling text-guided restoration and adjustable texture richness. Requiring only a single forward pass, it achieves faster convergence and inference speed than diffusion-based methods. Extensive experiments show that HYPIR outperforms previous state-of-the-art methods, achieving efficient and high-quality image restoration.

Related papers

• Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration [14.922600858354983]
  Diffusion Once and Done (DOD) method aims to achieve superior restoration performance with only one-step sampling of Stable Diffusion (SD) models.<n>Our method outperforms existing diffusion-based restoration approaches in both visual quality and inference efficiency.
  arXiv  Detail & Related papers  (2025-08-05T12:26:28Z)
• Quick Bypass Mechanism of Zero-Shot Diffusion-Based Image Restoration [0.8192907805418583]
  We propose a strategy that accelerates the denoising process by initializing from an intermediate approximation, effectively bypassing early denoising steps.<n>We validate proposed methods on ImageNet-1K and CelebAHQ across multiple image restoration tasks, e.g., super-resolution, deblurring, and compressed sensing.
  arXiv  Detail & Related papers  (2025-07-06T01:36:27Z)
• TD-BFR: Truncated Diffusion Model for Efficient Blind Face Restoration [17.79398314291093]
  We propose a novel Truncated Diffusion model for efficient Blind Face Restoration (TD-BFR)<n> TD-BFR utilizes an innovative truncated sampling method, starting from low-quality (LQ) images at low resolution to enhance sampling speed.<n>Our method efficiently restores high-quality images in a coarse-to-fine manner and experimental results demonstrate that TD-BFR is, on average, textbf4.75$times$ faster than current state-of-the-art diffusion-based BFR methods.
  arXiv  Detail & Related papers  (2025-03-26T13:35:43Z)
• Timestep-Aware Diffusion Model for Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling.<n>TADM performs rescaling operations in the latent space of a pre-trained autoencoder.<n>It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• ReNoise: Real Image Inversion Through Iterative Noising [62.96073631599749]
  We introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models.
  arXiv  Detail & Related papers  (2024-03-21T17:52:08Z)
• Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
  This study proposes a novel and efficient diffusion model for image restoration. Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration. Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
  arXiv  Detail & Related papers  (2024-03-12T05:06:07Z)
• PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance [65.5618804029422]
  Previous works have achieved noteworthy success by limiting the solution space using explicit degradation models. We propose PGDiff by introducing partial guidance, a fresh perspective that is more adaptable to real-world degradations. Our method not only outperforms existing diffusion-prior-based approaches but also competes favorably with task-specific models.
  arXiv  Detail & Related papers  (2023-09-19T17:51:33Z)
• Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey [96.99328714941657]
  We present a comprehensive review of recent diffusion model-based methods on image restoration. We classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR. We propose five potential and challenging directions for the future research of diffusion model-based IR.
  arXiv  Detail & Related papers  (2023-08-18T08:40:38Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.