PartDiff: Image Super-resolution with Partial Diffusion Models
- URL: http://arxiv.org/abs/2307.11926v1
- Date: Fri, 21 Jul 2023 22:11:23 GMT
- Title: PartDiff: Image Super-resolution with Partial Diffusion Models
- Authors: Kai Zhao, Alex Ling Yu Hung, Kaifeng Pang, Haoxin Zheng, and Kyunghyun
Sung
- Abstract summary: Denoising diffusion probabilistic models (DDPMs) have achieved impressive performance on various image generation tasks.
DDPMs generate new data by iteratively denoising from random noise.
But diffusion-based generative models suffer from high computational costs due to the large number of denoising steps.
This paper proposes the Partial Diffusion Model (PartDiff), which diffuses the image to an intermediate latent state instead of pure random noise.
- Score: 3.8435187580887717
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Denoising diffusion probabilistic models (DDPMs) have achieved impressive
performance on various image generation tasks, including image
super-resolution. By learning to reverse the process of gradually diffusing the
data distribution into Gaussian noise, DDPMs generate new data by iteratively
denoising from random noise. Despite their impressive performance,
diffusion-based generative models suffer from high computational costs due to
the large number of denoising steps.In this paper, we first observed that the
intermediate latent states gradually converge and become indistinguishable when
diffusing a pair of low- and high-resolution images. This observation inspired
us to propose the Partial Diffusion Model (PartDiff), which diffuses the image
to an intermediate latent state instead of pure random noise, where the
intermediate latent state is approximated by the latent of diffusing the
low-resolution image. During generation, Partial Diffusion Models start
denoising from the intermediate distribution and perform only a part of the
denoising steps. Additionally, to mitigate the error caused by the
approximation, we introduce "latent alignment", which aligns the latent between
low- and high-resolution images during training. Experiments on both magnetic
resonance imaging (MRI) and natural images show that, compared to plain
diffusion-based super-resolution methods, Partial Diffusion Models
significantly reduce the number of denoising steps without sacrificing the
quality of generation.
Related papers
- Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment [56.609042046176555]
Current methods diffuse each image across the entire noise space, resulting in a mixture of all images at every point in the noise layer.
We propose Immiscible Diffusion, a simple and effective method to improve the random mixture of noise-data mapping.
Our approach is remarkably simple, requiring only one line of code to restrict the diffuse-able area for each image.
arXiv Detail & Related papers (2024-06-18T06:20:42Z) - Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise [34.65659277870287]
Research on denoising diffusion models has expanded its application to the field of image restoration.
We propose Resfusion, a framework that incorporates the residual term into the diffusion forward process.
We show that Resfusion exhibits competitive performance on ISTD dataset, LOL dataset and Raindrop dataset with only five sampling steps.
arXiv Detail & Related papers (2023-11-25T02:09:38Z) - Denoising Diffusion Bridge Models [54.87947768074036]
Diffusion models are powerful generative models that map noise to data using processes.
For many applications such as image editing, the model input comes from a distribution that is not random noise.
In our work, we propose Denoising Diffusion Bridge Models (DDBMs)
arXiv Detail & Related papers (2023-09-29T03:24:24Z) - Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation.
We present GradPaint, which steers the generation towards a globally coherent image.
We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
arXiv Detail & Related papers (2023-09-18T09:36:24Z) - SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired
Image-to-Image Translation [96.11061713135385]
This work presents a new score-decomposed diffusion model to explicitly optimize the tangled distributions during image generation.
We equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold.
SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.
arXiv Detail & Related papers (2023-08-04T06:21:57Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - SVNR: Spatially-variant Noise Removal with Denoising Diffusion [43.2405873681083]
We present a novel formulation of denoising diffusion that assumes a more realistic, spatially-variant noise model.
In experiments we demonstrate the advantages of our approach over a strong diffusion model baseline, as well as over a state-of-the-art single image denoising method.
arXiv Detail & Related papers (2023-06-28T09:32:00Z) - A Variational Perspective on Solving Inverse Problems with Diffusion
Models [101.831766524264]
Inverse tasks can be formulated as inferring a posterior distribution over data.
This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable.
We propose a variational approach that by design seeks to approximate the true posterior distribution.
arXiv Detail & Related papers (2023-05-07T23:00:47Z) - CoreDiff: Contextual Error-Modulated Generalized Diffusion Model for
Low-Dose CT Denoising and Generalization [41.64072751889151]
Low-dose computed tomography (LDCT) images suffer from noise and artifacts due to photon starvation and electronic noise.
This paper presents a novel COntextual eRror-modulated gEneralized Diffusion model for low-dose CT (LDCT) denoising, termed CoreDiff.
arXiv Detail & Related papers (2023-04-04T14:13:13Z) - Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models
for Inverse Problems through Stochastic Contraction [31.61199061999173]
Diffusion models have a critical downside - they are inherently slow to sample from, needing few thousand steps of iteration to generate images from pure Gaussian noise.
We show that starting from Gaussian noise is unnecessary. Instead, starting from a single forward diffusion with better initialization significantly reduces the number of sampling steps in the reverse conditional diffusion.
New sampling strategy, dubbed ComeCloser-DiffuseFaster (CCDF), also reveals a new insight on how the existing feedforward neural network approaches for inverse problems can be synergistically combined with the diffusion models.
arXiv Detail & Related papers (2021-12-09T04:28:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.