Beta Diffusion

• URL: http://arxiv.org/abs/2309.07867v4
• Date: Mon, 25 Dec 2023 04:26:27 GMT
• Title: Beta Diffusion
• Authors: Mingyuan Zhou and Tianqi Chen and Zhendong Wang and Huangjie Zheng
• Abstract summary: We introduce beta diffusion, a novel generative modeling method that integrates demasking and denoising to generate data within bounded ranges. Beta diffusion is multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence. Experimental results on both synthetic data and natural images demonstrate the unique capabilities of beta diffusion in generative modeling of range-bounded data.
• Score: 69.61105403426778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce beta diffusion, a novel generative modeling method that integrates demasking and denoising to generate data within bounded ranges. Using scaled and shifted beta distributions, beta diffusion utilizes multiplicative transitions over time to create both forward and reverse diffusion processes, maintaining beta distributions in both the forward marginals and the reverse conditionals, given the data at any point in time. Unlike traditional diffusion-based generative models relying on additive Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence. We demonstrate that the proposed KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs, which can also be derived as the KLUBs of the same KL divergence with its two arguments swapped. The loss function of beta diffusion, expressed in terms of Bregman divergence, further supports the efficacy of KLUBs for optimization. Experimental results on both synthetic data and natural images demonstrate the unique capabilities of beta diffusion in generative modeling of range-bounded data and validate the effectiveness of KLUBs in optimizing diffusion models, thereby making them valuable additions to the family of diffusion-based generative models and the optimization techniques used to train them.

Related papers

• Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models [59.47572583027685]
  We provide the first finite-time error bounds and convergence rate analysis for discrete diffusion models using absorbing rate matrices.<n>We establish the first convergence guarantees for both the $tau$-leaping and uniformization samplers under absorbing rate matrices.<n>Under suitable assumptions, we provide convergence guarantees without early stopping.
  arXiv  Detail & Related papers  (2025-06-02T23:14:35Z)
• Forward-only Diffusion Probabilistic Models [14.538117998129307]
  This work presents a forward-only diffusion (FoD) approach for generative modelling.<n>FoD directly learns data generation through a single forward diffusion process.<n>FoD is analytically tractable and is trained using a simple flow matching objective.
  arXiv  Detail & Related papers  (2025-05-22T14:47:07Z)
• Exploring the flavor structure of leptons via diffusion models [0.0]
  We consider a simple extension of the Standard Model with the type I seesaw mechanism and train a neural network to generate the neutrino mass matrix. By utilizing transfer learning, the diffusion model generates 104 solutions consistent with the neutrino mass squared differences and the leptonic mixing angles.
  arXiv  Detail & Related papers  (2025-03-27T12:17:00Z)
• The Superposition of Diffusion Models Using the Itô Density Estimator [46.03684204456143]
  We show that SuperDiff is scalable to large pre-trained diffusion models. We also show that SuperDiff is efficient during inference time, and mimics traditional composition operators. We empirically demonstrate the utility of using SuperDiff for generating more diverse images on CIFAR-10.
  arXiv  Detail & Related papers  (2024-12-23T18:18:07Z)
• Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs [30.973473583364832]
  DoSSR is a Domain Shift diffusion-based SR model that capitalizes on the generative powers of pretrained diffusion models. At the core of our approach is a domain shift equation that integrates seamlessly with existing diffusion models. Our proposed method achieves state-of-the-art performance on synthetic and real-world datasets, while notably requiring only 5 sampling steps.
  arXiv  Detail & Related papers  (2024-09-26T12:16:11Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Fair Sampling in Diffusion Models through Switching Mechanism [5.560136885815622]
  We propose a fairness-aware sampling method called textitattribute switching mechanism for diffusion models. We mathematically prove and experimentally demonstrate the effectiveness of the proposed method on two key aspects.
  arXiv  Detail & Related papers  (2024-01-06T06:55:26Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• Eliminating Lipschitz Singularities in Diffusion Models [51.806899946775076]
  We show that diffusion models frequently exhibit the infinite Lipschitz near the zero point of timesteps. This poses a threat to the stability and accuracy of the diffusion process, which relies on integral operations. We propose a novel approach, dubbed E-TSDM, which eliminates the Lipschitz of the diffusion model near zero.
  arXiv  Detail & Related papers  (2023-06-20T03:05:28Z)
• Information-Theoretic Diffusion [18.356162596599436]
  Denoising diffusion models have spurred significant gains in density modeling and image generation. We introduce a new mathematical foundation for diffusion models inspired by classic results in information theory.
  arXiv  Detail & Related papers  (2023-02-07T23:03:07Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• Flexible Amortized Variational Inference in qBOLD MRI [56.4324135502282]
  Oxygen extraction fraction (OEF) and deoxygenated blood volume (DBV) are more ambiguously determined from the data. Existing inference methods tend to yield very noisy and underestimated OEF maps, while overestimating DBV. This work describes a novel probabilistic machine learning approach that can infer plausible distributions of OEF and DBV.
  arXiv  Detail & Related papers  (2022-03-11T10:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.