Improving and Unifying Discrete&Continuous-time Discrete Denoising
Diffusion
- URL: http://arxiv.org/abs/2402.03701v1
- Date: Tue, 6 Feb 2024 04:42:36 GMT
- Title: Improving and Unifying Discrete&Continuous-time Discrete Denoising
Diffusion
- Authors: Lingxiao Zhao, Xueying Ding, Lijun Yu, Leman Akoglu
- Abstract summary: We present a series of mathematical simplifications of the variational lower bound that enable more accurate and easy-to-optimize training for discrete diffusion.
We derive a simple formulation for backward denoising that enables exact and accelerated sampling, and importantly, an elegant unification of discrete-time and continuous-time discrete diffusion.
- Score: 41.03548068279262
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Discrete diffusion models have seen a surge of attention with applications on
naturally discrete data such as language and graphs. Although discrete-time
discrete diffusion has been established for a while, only recently Campbell et
al. (2022) introduced the first framework for continuous-time discrete
diffusion. However, their training and sampling processes differ significantly
from the discrete-time version, necessitating nontrivial approximations for
tractability. In this paper, we first present a series of mathematical
simplifications of the variational lower bound that enable more accurate and
easy-to-optimize training for discrete diffusion. In addition, we derive a
simple formulation for backward denoising that enables exact and accelerated
sampling, and importantly, an elegant unification of discrete-time and
continuous-time discrete diffusion. Thanks to simpler analytical formulations,
both forward and now also backward probabilities can flexibly accommodate any
noise distribution, including different noise distributions for multi-element
objects. Experiments show that our proposed USD3 (for Unified Simplified
Discrete Denoising Diffusion) outperform all SOTA baselines on established
datasets. We open-source our unified code at
https://github.com/LingxiaoShawn/USD3.
Related papers
- Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment [56.609042046176555]
Current methods diffuse each image across the entire noise space, resulting in a mixture of all images at every point in the noise layer.
We propose Immiscible Diffusion, a simple and effective method to improve the random mixture of noise-data mapping.
Our approach is remarkably simple, requiring only one line of code to restrict the diffuse-able area for each image.
arXiv Detail & Related papers (2024-06-18T06:20:42Z) - Non-asymptotic Convergence of Discrete-time Diffusion Models: New Approach and Improved Rate [49.97755400231656]
We establish convergence guarantees for substantially larger classes of distributions under DT diffusion processes.
We then specialize our results to a number of interesting classes of distributions with explicit parameter dependencies.
We propose a novel accelerated sampler and show that it improves the convergence rates of the corresponding regular sampler by orders of magnitude with respect to all system parameters.
arXiv Detail & Related papers (2024-02-21T16:11:47Z) - Multi-scale Diffusion Denoised Smoothing [79.95360025953931]
randomized smoothing has become one of a few tangible approaches that offers adversarial robustness to models at scale.
We present scalable methods to address the current trade-off between certified robustness and accuracy in denoised smoothing.
Our experiments show that the proposed multi-scale smoothing scheme combined with diffusion fine-tuning enables strong certified robustness available with high noise level.
arXiv Detail & Related papers (2023-10-25T17:11:21Z) - Mixed Variational Flows for Discrete Variables [14.00384446902181]
We develop a variational flow family for discrete distributions without any continuous embedding.
First, we develop a measure-preserving and discrete (MAD) invertible map that leaves the discrete target invariant.
We also develop an extension to MAD Mix that handles joint discrete and continuous models.
arXiv Detail & Related papers (2023-08-29T20:13:37Z) - Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces [0.0]
We develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process.
As an example, we introduce Blackout Diffusion'', which learns to produce samples from an empty image instead of from noise.
arXiv Detail & Related papers (2023-05-18T16:24:12Z) - DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises [38.72460741779243]
We introduce DINOISER to facilitate diffusion models for sequence generation by manipulating noises.
Experiments show that DINOISER enables consistent improvement over the baselines of previous diffusion-based sequence generative models.
arXiv Detail & Related papers (2023-02-20T15:14:46Z) - Diffusion-GAN: Training GANs with Diffusion [135.24433011977874]
Generative adversarial networks (GANs) are challenging to train stably.
We propose Diffusion-GAN, a novel GAN framework that leverages a forward diffusion chain to generate instance noise.
We show that Diffusion-GAN can produce more realistic images with higher stability and data efficiency than state-of-the-art GANs.
arXiv Detail & Related papers (2022-06-05T20:45:01Z) - Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial
Auto-Encoders [137.1060633388405]
Diffusion-based generative models learn how to generate the data by inferring a reverse diffusion chain.
We propose a faster and cheaper approach that adds noise not until the data become pure random noise.
We show that the proposed model can be cast as an adversarial auto-encoder empowered by both the diffusion process and a learnable implicit prior.
arXiv Detail & Related papers (2022-02-19T20:18:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.