Beyond Fixed Horizons: A Theoretical Framework for Adaptive Denoising Diffusions

• URL: http://arxiv.org/abs/2501.19373v1
• Date: Fri, 31 Jan 2025 18:23:27 GMT
• Title: Beyond Fixed Horizons: A Theoretical Framework for Adaptive Denoising Diffusions
• Authors: Sören Christensen, Claudia Strauch, Lukas Trottner,
• Abstract summary: We introduce a new class of generative diffusion models that achieve a time-homogeneous structure for both the noising and denoising processes. A key feature of the model is its adaptability to the target data, enabling a variety of downstream tasks using a pre-trained unconditional generative model.
• Score: 1.9116784879310031
• License:
• Abstract: We introduce a new class of generative diffusion models that, unlike conventional denoising diffusion models, achieve a time-homogeneous structure for both the noising and denoising processes, allowing the number of steps to adaptively adjust based on the noise level. This is accomplished by conditioning the forward process using Doob's $h$-transform, which terminates the process at a suitable sampling distribution at a random time. The model is particularly well suited for generating data with lower intrinsic dimensions, as the termination criterion simplifies to a first-hitting rule. A key feature of the model is its adaptability to the target data, enabling a variety of downstream tasks using a pre-trained unconditional generative model. These tasks include natural conditioning through appropriate initialization of the denoising process and classification of noisy data.

Related papers

• A First-order Generative Bilevel Optimization Framework for Diffusion Models [57.40597004445473]
  Diffusion models iteratively denoise data samples to synthesize high-quality outputs. Traditional bilevel methods fail due to infinite-dimensional probability space and prohibitive sampling costs. We formalize this challenge as a generative bilevel optimization problem. Our first-order bilevel framework overcomes the incompatibility of conventional bilevel methods with diffusion processes.
  arXiv  Detail & Related papers  (2025-02-12T21:44:06Z)
• Generative diffusion model with inverse renormalization group flows [0.0]
  Diffusion models produce data by denoising a sample corrupted by white noise. We introduce a renormalization group-based diffusion model that leverages multiscale nature of data distributions. We validate the versatility of the model through applications to protein structure prediction and image generation.
  arXiv  Detail & Related papers  (2025-01-15T19:00:01Z)
• Learning with Noisy Foundation Models [95.50968225050012]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets. We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Latent Class-Conditional Noise Model [54.56899309997246]
  We introduce a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples.
  arXiv  Detail & Related papers  (2023-02-19T15:24:37Z)
• Entropy-driven Sampling and Training Scheme for Conditional Diffusion Generation [16.13197951857033]
  Denoising Diffusion Probabilistic Model (DDPM) is able to make flexible conditional image generation from prior noise to real data. However, due to the ability of classifier to easily discriminate an incompletely generated image only with high-level structure, the gradient tends to vanish early. We propose two simple but effective approaches from two perspectives to address this problem.
  arXiv  Detail & Related papers  (2022-06-23T04:10:23Z)
• PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Driven Adaptive Prior [103.00403682863427]
  We propose PriorGrad to improve the efficiency of the conditional diffusion model. We show that PriorGrad achieves a faster convergence leading to data and parameter efficiency and improved quality.
  arXiv  Detail & Related papers  (2021-06-11T14:04:03Z)
• Set Based Stochastic Subsampling [85.5331107565578]
  We propose a set-based two-stage end-to-end neural subsampling model that is jointly optimized with an textitarbitrary downstream task network. We show that it outperforms the relevant baselines under low subsampling rates on a variety of tasks including image classification, image reconstruction, function reconstruction and few-shot classification.
  arXiv  Detail & Related papers  (2020-06-25T07:36:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.