Accelerating Score-based Generative Models for High-Resolution Image
Synthesis
- URL: http://arxiv.org/abs/2206.04029v3
- Date: Fri, 10 Jun 2022 08:53:54 GMT
- Title: Accelerating Score-based Generative Models for High-Resolution Image
Synthesis
- Authors: Hengyuan Ma, Li Zhang, Xiatian Zhu, Jingfeng Zhang, Jianfeng Feng
- Abstract summary: Score-based generative models (SGMs) have recently emerged as a promising class of generative models.
In this work, we consider the acceleration of high-resolution generation with SGMs.
We introduce a novel Target Distribution Sampling Aware (TDAS) method by leveraging the structural priors in space and frequency domains.
- Score: 42.076244561541706
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Score-based generative models (SGMs) have recently emerged as a promising
class of generative models. The key idea is to produce high-quality images by
recurrently adding Gaussian noises and gradients to a Gaussian sample until
converging to the target distribution, a.k.a. the diffusion sampling. To ensure
stability of convergence in sampling and generation quality, however, this
sequential sampling process has to take a small step size and many sampling
iterations (e.g., 2000). Several acceleration methods have been proposed with
focus on low-resolution generation. In this work, we consider the acceleration
of high-resolution generation with SGMs, a more challenging yet more important
problem. We prove theoretically that this slow convergence drawback is
primarily due to the ignorance of the target distribution. Further, we
introduce a novel Target Distribution Aware Sampling (TDAS) method by
leveraging the structural priors in space and frequency domains. Extensive
experiments on CIFAR-10, CelebA, LSUN, and FFHQ datasets validate that TDAS can
consistently accelerate state-of-the-art SGMs, particularly on more challenging
high resolution (1024x1024) image generation tasks by up to 18.4x, whilst
largely maintaining the synthesis quality. With fewer sampling iterations, TDAS
can still generate good quality images. In contrast, the existing methods
degrade drastically or even fails completely
Related papers
- Score-based Generative Models with Adaptive Momentum [40.84399531998246]
We propose an adaptive momentum sampling method to accelerate the transforming process.
We show that our method can produce more faithful images/graphs in small sampling steps with 2 to 5 times speed up.
arXiv Detail & Related papers (2024-05-22T15:20:27Z) - SinSR: Diffusion-Based Image Super-Resolution in a Single Step [119.18813219518042]
Super-resolution (SR) methods based on diffusion models exhibit promising results.
But their practical application is hindered by the substantial number of required inference steps.
We propose a simple yet effective method for achieving single-step SR generation, named SinSR.
arXiv Detail & Related papers (2023-11-23T16:21:29Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Simultaneous Image-to-Zero and Zero-to-Noise: Diffusion Models with Analytical Image Attenuation [53.04220377034574]
We propose incorporating an analytical image attenuation process into the forward diffusion process for high-quality (un)conditioned image generation.
Our method represents the forward image-to-noise mapping as simultaneous textitimage-to-zero mapping and textitzero-to-noise mapping.
We have conducted experiments on unconditioned image generation, textite.g., CIFAR-10 and CelebA-HQ-256, and image-conditioned downstream tasks such as super-resolution, saliency detection, edge detection, and image inpainting.
arXiv Detail & Related papers (2023-06-23T18:08:00Z) - Preconditioned Score-based Generative Models [49.88840603798831]
An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation.
We propose a model-agnostic bfem preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem.
PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining.
arXiv Detail & Related papers (2023-02-13T16:30:53Z) - Deep Equilibrium Approaches to Diffusion Models [1.4275201654498746]
Diffusion-based generative models are extremely effective in generating high-quality images.
These models typically require long sampling chains to produce high-fidelity images.
We look at diffusion models through a different perspective, that of a (deep) equilibrium (DEQ) fixed point model.
arXiv Detail & Related papers (2022-10-23T22:02:19Z) - Accelerating Score-based Generative Models with Preconditioned Diffusion
Sampling [36.02321871608158]
We propose a model-agnostic preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the problem.
PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality.
In particular, PDS can accelerate by up to 29x on more challenging high resolution (1024x1024) image generation.
arXiv Detail & Related papers (2022-07-05T17:55:42Z) - Unsupervised Single Image Super-resolution Under Complex Noise [60.566471567837574]
This paper proposes a model-based unsupervised SISR method to deal with the general SISR task with unknown degradations.
The proposed method can evidently surpass the current state of the art (SotA) method (about 1dB PSNR) not only with a slighter model (0.34M vs. 2.40M) but also faster speed.
arXiv Detail & Related papers (2021-07-02T11:55:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.