A Complete Recipe for Diffusion Generative Models
- URL: http://arxiv.org/abs/2303.01748v2
- Date: Wed, 11 Oct 2023 20:28:58 GMT
- Title: A Complete Recipe for Diffusion Generative Models
- Authors: Kushagra Pandey, Stephan Mandt
- Abstract summary: We present a complete recipe for formulating forward processes in Generative Models (SGMs)
We introduce Phase Space Langevin Diffusion (PSLD), which relies on score-based modeling within an augmented space enriched by auxiliary variables.
- Score: 18.891215475887314
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Score-based Generative Models (SGMs) have demonstrated exceptional synthesis
outcomes across various tasks. However, the current design landscape of the
forward diffusion process remains largely untapped and often relies on physical
heuristics or simplifying assumptions. Utilizing insights from the development
of scalable Bayesian posterior samplers, we present a complete recipe for
formulating forward processes in SGMs, ensuring convergence to the desired
target distribution. Our approach reveals that several existing SGMs can be
seen as specific manifestations of our framework. Building upon this method, we
introduce Phase Space Langevin Diffusion (PSLD), which relies on score-based
modeling within an augmented space enriched by auxiliary variables akin to
physical phase space. Empirical results exhibit the superior sample quality and
improved speed-quality trade-off of PSLD compared to various competing
approaches on established image synthesis benchmarks. Remarkably, PSLD achieves
sample quality akin to state-of-the-art SGMs (FID: 2.10 for unconditional
CIFAR-10 generation). Lastly, we demonstrate the applicability of PSLD in
conditional synthesis using pre-trained score networks, offering an appealing
alternative as an SGM backbone for future advancements. Code and model
checkpoints can be accessed at \url{https://github.com/mandt-lab/PSLD}.
Related papers
- Synthetic location trajectory generation using categorical diffusion
models [50.809683239937584]
Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data.
We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
arXiv Detail & Related papers (2024-02-19T15:57:39Z) - Wasserstein Convergence Guarantees for a General Class of Score-Based
Generative Models [9.47767039367222]
Score-based generative models (SGMs) are a recent class of deep generative models with state-of-the-art performance in many applications.
We establish convergence guarantees for a general class of SGMs in 2-Wasserstein distance, assuming accurate score estimates and smooth log-concave data distribution.
Numerically, we experiment SGMs with different forward processes, some of which are newly proposed in this paper, for unconditional image generation on CIFAR-10.
arXiv Detail & Related papers (2023-11-18T07:53:22Z) - Protein Design with Guided Discrete Diffusion [67.06148688398677]
A popular approach to protein design is to combine a generative model with a discriminative model for conditional sampling.
We propose diffusioN Optimized Sampling (NOS), a guidance method for discrete diffusion models.
NOS makes it possible to perform design directly in sequence space, circumventing significant limitations of structure-based methods.
arXiv Detail & Related papers (2023-05-31T16:31:24Z) - Preconditioned Score-based Generative Models [49.88840603798831]
An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation.
We propose a model-agnostic bfem preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem.
PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining.
arXiv Detail & Related papers (2023-02-13T16:30:53Z) - A new perspective on probabilistic image modeling [92.89846887298852]
We present a new probabilistic approach for image modeling capable of density estimation, sampling and tractable inference.
DCGMMs can be trained end-to-end by SGD from random initial conditions, much like CNNs.
We show that DCGMMs compare favorably to several recent PC and SPN models in terms of inference, classification and sampling.
arXiv Detail & Related papers (2022-03-21T14:53:57Z) - DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from
Low-Dimensional Latents [26.17940552906923]
We present DiffuseVAE, a novel generative framework that integrates VAE within a diffusion model framework.
We show that the proposed model can generate high-resolution samples and exhibits quality comparable to state-of-the-art models on standard benchmarks.
arXiv Detail & Related papers (2022-01-02T06:44:23Z) - Score-Based Generative Modeling with Critically-Damped Langevin
Diffusion [18.82116696636531]
Current score-based generative models (SGMs) rely on a diffusion process that gradually perturbs the data towards a tractable distribution.
We argue that current SGMs employ overly simplistic diffusions, leading to unnecessarily complex denoising processes.
We propose a novel critically-damped Langevin diffusion (CLD) and show that CLD-based SGMs achieve superior performance.
arXiv Detail & Related papers (2021-12-14T00:01:34Z) - Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems.
Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models.
We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
arXiv Detail & Related papers (2021-11-24T08:03:48Z) - Score-based Generative Modeling in Latent Space [93.8985523558869]
Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution coverage.
Here, we propose the Latent Score-based Generative Model (LSGM), a novel approach that trains SGMs in a latent space.
Moving from data to latent space allows us to train more expressive generative models, apply SGMs to non-continuous data, and learn smoother SGMs in a smaller space.
arXiv Detail & Related papers (2021-06-10T17:26:35Z) - Elastic Consistency: A General Consistency Model for Distributed
Stochastic Gradient Descent [28.006781039853575]
A key element behind the progress of machine learning in recent years has been the ability to train machine learning models in largescale distributed-memory environments.
In this paper, we introduce general convergence methods used in practice to train large-scale machine learning models.
Our framework, called elastic elastic bounds, enables us to derive convergence bounds for a variety of distributed SGD methods.
arXiv Detail & Related papers (2020-01-16T16:10:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.