ResShift: Efficient Diffusion Model for Image Super-resolution by
Residual Shifting
- URL: http://arxiv.org/abs/2307.12348v3
- Date: Wed, 18 Oct 2023 11:50:49 GMT
- Title: ResShift: Efficient Diffusion Model for Image Super-resolution by
Residual Shifting
- Authors: Zongsheng Yue, Jianyi Wang, Chen Change Loy
- Abstract summary: Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed.
We propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps.
Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual.
- Score: 70.83632337581034
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Diffusion-based image super-resolution (SR) methods are mainly limited by the
low inference speed due to the requirements of hundreds or even thousands of
sampling steps. Existing acceleration sampling techniques inevitably sacrifice
performance to some extent, leading to over-blurry SR results. To address this
issue, we propose a novel and efficient diffusion model for SR that
significantly reduces the number of diffusion steps, thereby eliminating the
need for post-acceleration during inference and its associated performance
deterioration. Our method constructs a Markov chain that transfers between the
high-resolution image and the low-resolution image by shifting the residual
between them, substantially improving the transition efficiency. Additionally,
an elaborate noise schedule is developed to flexibly control the shifting speed
and the noise strength during the diffusion process. Extensive experiments
demonstrate that the proposed method obtains superior or at least comparable
performance to current state-of-the-art methods on both synthetic and
real-world datasets, even only with 15 sampling steps. Our code and model are
available at https://github.com/zsyOAOA/ResShift.
Related papers
- Distilling Diffusion Models into Conditional GANs [90.76040478677609]
We distill a complex multistep diffusion model into a single-step conditional GAN student model.
For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space.
We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
arXiv Detail & Related papers (2024-05-09T17:59:40Z) - Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
This study proposes a novel and efficient diffusion model for image restoration.
Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration.
Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
arXiv Detail & Related papers (2024-03-12T05:06:07Z) - SinSR: Diffusion-Based Image Super-Resolution in a Single Step [119.18813219518042]
Super-resolution (SR) methods based on diffusion models exhibit promising results.
But their practical application is hindered by the substantial number of required inference steps.
We propose a simple yet effective method for achieving single-step SR generation, named SinSR.
arXiv Detail & Related papers (2023-11-23T16:21:29Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration.
We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
arXiv Detail & Related papers (2023-06-01T03:08:28Z) - Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models
for Inverse Problems through Stochastic Contraction [31.61199061999173]
Diffusion models have a critical downside - they are inherently slow to sample from, needing few thousand steps of iteration to generate images from pure Gaussian noise.
We show that starting from Gaussian noise is unnecessary. Instead, starting from a single forward diffusion with better initialization significantly reduces the number of sampling steps in the reverse conditional diffusion.
New sampling strategy, dubbed ComeCloser-DiffuseFaster (CCDF), also reveals a new insight on how the existing feedforward neural network approaches for inverse problems can be synergistically combined with the diffusion models.
arXiv Detail & Related papers (2021-12-09T04:28:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.