Efficient Diffusion Model for Image Restoration by Residual Shifting
- URL: http://arxiv.org/abs/2403.07319v1
- Date: Tue, 12 Mar 2024 05:06:07 GMT
- Title: Efficient Diffusion Model for Image Restoration by Residual Shifting
- Authors: Zongsheng Yue, Jianyi Wang, and Chen Change Loy
- Abstract summary: This study proposes a novel and efficient diffusion model for image restoration.
Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration.
Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
- Score: 63.02725947015132
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: While diffusion-based image restoration (IR) methods have achieved remarkable
success, they are still limited by the low inference speed attributed to the
necessity of executing hundreds or even thousands of sampling steps. Existing
acceleration sampling techniques, though seeking to expedite the process,
inevitably sacrifice performance to some extent, resulting in over-blurry
restored outcomes. To address this issue, this study proposes a novel and
efficient diffusion model for IR that significantly reduces the required number
of diffusion steps. Our method avoids the need for post-acceleration during
inference, thereby avoiding the associated performance deterioration.
Specifically, our proposed method establishes a Markov chain that facilitates
the transitions between the high-quality and low-quality images by shifting
their residuals, substantially improving the transition efficiency. A carefully
formulated noise schedule is devised to flexibly control the shifting speed and
the noise strength during the diffusion process. Extensive experimental
evaluations demonstrate that the proposed method achieves superior or
comparable performance to current state-of-the-art methods on three classical
IR tasks, namely image super-resolution, image inpainting, and blind face
restoration, \textit{\textbf{even only with four sampling steps}}. Our code and
model are publicly available at \url{https://github.com/zsyOAOA/ResShift}.
Related papers
- ReNoise: Real Image Inversion Through Iterative Noising [62.96073631599749]
We introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations.
We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models.
arXiv Detail & Related papers (2024-03-21T17:52:08Z) - Implicit Image-to-Image Schrodinger Bridge for CT Super-Resolution and
Denoising [13.529438499545408]
The Image-to-Image Schr"odinger Bridge (I2SB) initializes the generative process from corrupted images.
We introduce the Implicit Image-to-Image Schrodinger Bridge (I3SB), transitioning its generative process to a non-Markovian process.
This enhancement empowers I3SB to generate images with better texture restoration using a small number of generative steps.
arXiv Detail & Related papers (2024-03-10T03:22:57Z) - SinSR: Diffusion-Based Image Super-Resolution in a Single Step [119.18813219518042]
Super-resolution (SR) methods based on diffusion models exhibit promising results.
But their practical application is hindered by the substantial number of required inference steps.
We propose a simple yet effective method for achieving single-step SR generation, named SinSR.
arXiv Detail & Related papers (2023-11-23T16:21:29Z) - ResShift: Efficient Diffusion Model for Image Super-resolution by
Residual Shifting [70.83632337581034]
Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed.
We propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps.
Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual.
arXiv Detail & Related papers (2023-07-23T15:10:02Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration.
We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
arXiv Detail & Related papers (2023-06-01T03:08:28Z) - Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models
for Inverse Problems through Stochastic Contraction [31.61199061999173]
Diffusion models have a critical downside - they are inherently slow to sample from, needing few thousand steps of iteration to generate images from pure Gaussian noise.
We show that starting from Gaussian noise is unnecessary. Instead, starting from a single forward diffusion with better initialization significantly reduces the number of sampling steps in the reverse conditional diffusion.
New sampling strategy, dubbed ComeCloser-DiffuseFaster (CCDF), also reveals a new insight on how the existing feedforward neural network approaches for inverse problems can be synergistically combined with the diffusion models.
arXiv Detail & Related papers (2021-12-09T04:28:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.