Relay Diffusion: Unifying diffusion process across resolutions for image
synthesis
- URL: http://arxiv.org/abs/2309.03350v1
- Date: Mon, 4 Sep 2023 15:00:33 GMT
- Title: Relay Diffusion: Unifying diffusion process across resolutions for image
synthesis
- Authors: Jiayan Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni,
Zhuoyi Yang, Jie Tang
- Abstract summary: Relay Diffusion Model (RDM) transfers a low-resolution image or noise into an equivalent high-resolution one for diffusion model via blurring diffusion and block noise.
RDM achieves state-of-the-art FID on CelebA-HQ and sFID on ImageNet 256$times $256, surpassing previous works such as ADM, LDM and DiT by a large margin.
- Score: 26.96575808522695
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Diffusion models achieved great success in image synthesis, but still face
challenges in high-resolution generation. Through the lens of discrete cosine
transformation, we find the main reason is that \emph{the same noise level on a
higher resolution results in a higher Signal-to-Noise Ratio in the frequency
domain}. In this work, we present Relay Diffusion Model (RDM), which transfers
a low-resolution image or noise into an equivalent high-resolution one for
diffusion model via blurring diffusion and block noise. Therefore, the
diffusion process can continue seamlessly in any new resolution or model
without restarting from pure noise or low-resolution conditioning. RDM achieves
state-of-the-art FID on CelebA-HQ and sFID on ImageNet 256$\times$256,
surpassing previous works such as ADM, LDM and DiT by a large margin. All the
codes and checkpoints are open-sourced at
\url{https://github.com/THUDM/RelayDiffusion}.
Related papers
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution [11.326598938246558]
We propose a one-step effective diffusion network, namely OSEDiff, for the Real-ISR problem.
We apply variational score distillation in the latent space to conduct KL-divergence regularization.
Our experiments demonstrate that OSEDiff achieves comparable or even better Real-ISR results, in terms of both objective metrics and subjective evaluations.
arXiv Detail & Related papers (2024-06-12T13:10:31Z) - Diffusion-Aided Joint Source Channel Coding For High Realism Wireless Image Transmission [24.372996233209854]
DiffJSCC is a novel framework that produces high-realism images via the conditional diffusion denoising process.
It can achieve highly realistic reconstructions for 768x512 pixel Kodak images with only 3072 symbols.
arXiv Detail & Related papers (2024-04-27T00:12:13Z) - Domain Transfer in Latent Space (DTLS) Wins on Image Super-Resolution --
a Non-Denoising Model [13.326634982790528]
We propose a simple approach which gets away from using Gaussian noise but adopts some basic structures of diffusion models for efficient image super-resolution.
Experimental results show that our method outperforms not only state-of-the-art large scale super resolution models, but also the current diffusion models for image super-resolution.
arXiv Detail & Related papers (2023-11-04T09:57:50Z) - PartDiff: Image Super-resolution with Partial Diffusion Models [3.8435187580887717]
Denoising diffusion probabilistic models (DDPMs) have achieved impressive performance on various image generation tasks.
DDPMs generate new data by iteratively denoising from random noise.
But diffusion-based generative models suffer from high computational costs due to the large number of denoising steps.
This paper proposes the Partial Diffusion Model (PartDiff), which diffuses the image to an intermediate latent state instead of pure random noise.
arXiv Detail & Related papers (2023-07-21T22:11:23Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance.
We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring.
Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T12:18:20Z) - Denoising Diffusion Models for Plug-and-Play Image Restoration [135.6359475784627]
This paper proposes DiffPIR, which integrates the traditional plug-and-play method into the diffusion sampling framework.
Compared to plug-and-play IR methods that rely on discriminative Gaussian denoisers, DiffPIR is expected to inherit the generative ability of diffusion models.
arXiv Detail & Related papers (2023-05-15T20:24:38Z) - VideoFusion: Decomposed Diffusion Models for High-Quality Video
Generation [88.49030739715701]
This work presents a decomposed diffusion process via resolving the per-frame noise into a base noise that is shared among all frames and a residual noise that varies along the time axis.
Experiments on various datasets confirm that our approach, termed as VideoFusion, surpasses both GAN-based and diffusion-based alternatives in high-quality video generation.
arXiv Detail & Related papers (2023-03-15T02:16:39Z) - SDM: Spatial Diffusion Model for Large Hole Image Inpainting [106.90795513361498]
We present a novel spatial diffusion model (SDM) that uses a few iterations to gradually deliver informative pixels to the entire image.
Also, thanks to the proposed decoupled probabilistic modeling and spatial diffusion scheme, our method achieves high-quality large-hole completion.
arXiv Detail & Related papers (2022-12-06T13:30:18Z) - Dimensionality-Varying Diffusion Process [52.52681373641533]
Diffusion models learn to reverse a signal destruction process to generate new data.
We make a theoretical generalization of the forward diffusion process via signal decomposition.
We show that our strategy facilitates high-resolution image synthesis and improves FID of diffusion model trained on FFHQ at $1024times1024$ resolution from 52.40 to 10.46.
arXiv Detail & Related papers (2022-11-29T09:05:55Z) - Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image
Synthesis [39.671396431940224]
diffusion models have shown remarkable results in image synthesis by gradually removing noise and amplifying signals.
We propose a novel generative process that synthesizes images in a coarse-to-fine manner.
Experiments show that the proposed model outperforms the previous method in FID on LSUN bedroom and church datasets.
arXiv Detail & Related papers (2022-07-16T15:00:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.