Star-Shaped Denoising Diffusion Probabilistic Models
- URL: http://arxiv.org/abs/2302.05259v3
- Date: Sat, 28 Oct 2023 21:50:10 GMT
- Title: Star-Shaped Denoising Diffusion Probabilistic Models
- Authors: Andrey Okhotin, Dmitry Molchanov, Vladimir Arkhipkin, Grigory Bartosh,
Viktor Ohanesian, Aibek Alanov, Dmitry Vetrov
- Abstract summary: We introduce Star-Shaped DDPM (SSDDPM)
Our implementation is available at https://github.com/andreyokhotin/star-shaped.
- Score: 5.167803438665587
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Denoising Diffusion Probabilistic Models (DDPMs) provide the foundation for
the recent breakthroughs in generative modeling. Their Markovian structure
makes it difficult to define DDPMs with distributions other than Gaussian or
discrete. In this paper, we introduce Star-Shaped DDPM (SS-DDPM). Its
star-shaped diffusion process allows us to bypass the need to define the
transition probabilities or compute posteriors. We establish duality between
star-shaped and specific Markovian diffusions for the exponential family of
distributions and derive efficient algorithms for training and sampling from
SS-DDPMs. In the case of Gaussian distributions, SS-DDPM is equivalent to DDPM.
However, SS-DDPMs provide a simple recipe for designing diffusion models with
distributions such as Beta, von Mises$\unicode{x2013}$Fisher, Dirichlet,
Wishart and others, which can be especially useful when data lies on a
constrained manifold. We evaluate the model in different settings and find it
competitive even on image data, where Beta SS-DDPM achieves results comparable
to a Gaussian DDPM. Our implementation is available at
https://github.com/andrey-okhotin/star-shaped .
Related papers
- Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data [55.54827581105283]
We show that the concrete score in absorbing diffusion can be expressed as conditional probabilities of clean data.
We propose a dedicated diffusion model without time-condition that characterizes the time-independent conditional probabilities.
Our models achieve SOTA performance among diffusion models on 5 zero-shot language modeling benchmarks.
arXiv Detail & Related papers (2024-06-06T04:22:11Z) - Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations [54.95457207525101]
Denoising Diffusion Probabilistic Models (DDPMs) exhibit remarkable capabilities in image generation.
We study DDPMs trained on strictly separate subsets of the data distribution with large gaps on the support of latent factors.
We show that such a model can effectively generate images in the unexplored, intermediate regions of the distribution.
arXiv Detail & Related papers (2024-05-29T15:41:53Z) - DPM-OT: A New Diffusion Probabilistic Model Based on Optimal Transport [26.713392774427653]
DPM-OT is a unified learning framework for fast DPMs with a direct expressway represented by OT map.
It can generate high-quality samples within around 10 function evaluations.
Experiments validate the effectiveness and advantages of DPM-OT in terms of speed and quality.
arXiv Detail & Related papers (2023-07-21T02:28:54Z) - UDPM: Upsampling Diffusion Probabilistic Models [33.51145642279836]
Denoising Diffusion Probabilistic Models (DDPM) have recently gained significant attention.
DDPMs generate high-quality samples from complex data distributions by defining an inverse process.
Unlike generative adversarial networks (GANs), the latent space of diffusion models is less interpretable.
In this work, we propose to generalize the denoising diffusion process into an Upsampling Diffusion Probabilistic Model (UDPM)
arXiv Detail & Related papers (2023-05-25T17:25:14Z) - Fast Diffusion Probabilistic Model Sampling through the lens of Backward
Error Analysis [26.907301901503835]
Denoising diffusion probabilistic models (DDPMs) are a class of powerful generative models.
DDPMs generally need hundreds or thousands of sequential function evaluations (steps) of neural networks to generate a sample.
This paper aims to develop a fast sampling method for DDPMs requiring much fewer steps while retaining high sample quality.
arXiv Detail & Related papers (2023-04-22T16:58:47Z) - On Calibrating Diffusion Probabilistic Models [78.75538484265292]
diffusion probabilistic models (DPMs) have achieved promising results in diverse generative tasks.
We propose a simple way for calibrating an arbitrary pretrained DPM, with which the score matching loss can be reduced and the lower bounds of model likelihood can be increased.
Our calibration method is performed only once and the resulting models can be used repeatedly for sampling.
arXiv Detail & Related papers (2023-02-21T14:14:40Z) - Accelerating Diffusion Models via Early Stop of the Diffusion Process [114.48426684994179]
Denoising Diffusion Probabilistic Models (DDPMs) have achieved impressive performance on various generation tasks.
In practice DDPMs often need hundreds even thousands of denoising steps to obtain a high-quality sample.
We propose a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), for DDPMs.
arXiv Detail & Related papers (2022-05-25T06:40:09Z) - Pseudo Numerical Methods for Diffusion Models on Manifolds [77.40343577960712]
Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples.
DDPMs require hundreds to thousands of iterations to produce final samples.
We propose pseudo numerical methods for diffusion models (PNDMs)
PNDMs can generate higher quality synthetic images with only 50 steps compared with 1000-step DDIMs (20x speedup)
arXiv Detail & Related papers (2022-02-20T10:37:52Z) - Improved Denoising Diffusion Probabilistic Models [4.919647298882951]
We show that DDPMs can achieve competitive log-likelihoods while maintaining high sample quality.
We also find that learning variances of the reverse diffusion process allows sampling with an order of magnitude fewer forward passes.
We show that the sample quality and likelihood of these models scale smoothly with model capacity and training compute, making them easily scalable.
arXiv Detail & Related papers (2021-02-18T23:44:17Z) - Denoising Diffusion Implicit Models [117.03720513930335]
We present denoising diffusion implicit models (DDIMs) for iterative implicit probabilistic models with the same training procedure as DDPMs.
DDIMs can produce high quality samples $10 times$ to $50 times$ faster in terms of wall-clock time compared to DDPMs.
arXiv Detail & Related papers (2020-10-06T06:15:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.