Generative modelling with jump-diffusions

• URL: http://arxiv.org/abs/2503.06558v1
• Date: Sun, 09 Mar 2025 11:08:03 GMT
• Title: Generative modelling with jump-diffusions
• Authors: Adrian Baule,
• Abstract summary: I present a generalization of generative diffusion processes to a wide class of non-Gaussian noise processes.<n>For the problem of capturing a heavy-tailed target distribution, the jump-diffusion Laplace model outperforms models driven by alpha-stable noise.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Score-based diffusion models generate samples from an unknown target distribution using a time-reversed diffusion process. While such models represent state-of-the-art approaches in industrial applications such as artificial image generation, it has recently been noted that their performance can be further improved by considering injection noise with heavy tailed characteristics. Here, I present a generalization of generative diffusion processes to a wide class of non-Gaussian noise processes. I consider forward processes driven by standard Gaussian noise with super-imposed Poisson jumps representing a finite activity Levy process. The generative process is shown to be governed by a generalized score function that depends on the jump amplitude distribution. Both probability flow ODE and SDE formulations are derived using basic technical effort, and are implemented for jump amplitudes drawn from a multivariate Laplace distribution. Remarkably, for the problem of capturing a heavy-tailed target distribution, the jump-diffusion Laplace model outperforms models driven by alpha-stable noise despite not containing any heavy-tailed characteristics. The framework can be readily applied to other jump statistics that could further improve on the performance of standard diffusion models.

Related papers

• Critical Iterative Denoising: A Discrete Generative Model Applied to Graphs [52.50288418639075]
  We propose a novel framework called Iterative Denoising, which simplifies discrete diffusion and circumvents the issue by assuming conditional independence across time. Our empirical evaluations demonstrate that the proposed method significantly outperforms existing discrete diffusion baselines in graph generation tasks.
  arXiv  Detail & Related papers  (2025-03-27T15:08:58Z)
• Accelerated Diffusion Models via Speculative Sampling [89.43940130493233]
  Speculative sampling is a popular technique for accelerating inference in Large Language Models.<n>We extend speculative sampling to diffusion models, which generate samples via continuous, vector-valued Markov chains.<n>We propose various drafting strategies, including a simple and effective approach that does not require training a draft model.
  arXiv  Detail & Related papers  (2025-01-09T16:50:16Z)
• RDPM: Solve Diffusion Probabilistic Models via Recurrent Token Prediction [17.005198258689035]
  Diffusion Probabilistic Models (DPMs) have emerged as the de facto approach for high-fidelity image synthesis.<n>We introduce a novel generative framework, the Recurrent Diffusion Probabilistic Model (RDPM), which enhances the diffusion process through a recurrent token prediction mechanism.
  arXiv  Detail & Related papers  (2024-12-24T12:28:19Z)
• Arbitrary-steps Image Super-resolution via Diffusion Inversion [68.78628844966019]
  This study presents a new image super-resolution (SR) technique based on diffusion inversion, aiming at harnessing the rich image priors encapsulated in large pre-trained diffusion models to improve SR performance.<n>We design a Partial noise Prediction strategy to construct an intermediate state of the diffusion model, which serves as the starting sampling point.<n>Once trained, this noise predictor can be used to initialize the sampling process partially along the diffusion trajectory, generating the desirable high-resolution result.
  arXiv  Detail & Related papers  (2024-12-12T07:24:13Z)
• Non-Normal Diffusion Models [3.5534933448684134]
  Diffusion models generate samples by incrementally reversing a process that turns data into noise.<n>We show that when the step size goes to zero, the reversed process is invariant to the distribution of these increments.<n>We demonstrate the effectiveness of these models on density estimation and generative modeling tasks on standard image datasets.
  arXiv  Detail & Related papers  (2024-12-10T21:31:12Z)
• Generalized Diffusion Model with Adjusted Offset Noise [1.7767466724342067]
  We propose a generalized diffusion model that naturally incorporates additional noise within a rigorous probabilistic framework. We derive a loss function based on the evidence lower bound, establishing its theoretical equivalence to offset noise with certain adjustments. Experiments on synthetic datasets demonstrate that our model effectively addresses brightness-related challenges and outperforms conventional methods in high-dimensional scenarios.
  arXiv  Detail & Related papers  (2024-12-04T08:57:03Z)
• A prior regularized full waveform inversion using generative diffusion models [0.5156484100374059]
  Full waveform inversion (FWI) has the potential to provide high-resolution subsurface model estimations. Due to limitations in observation, e.g., regional noise, limited shots or receivers, and band-limited data, it is hard to obtain the desired high-resolution model with FWI. We propose a new paradigm for FWI regularized by generative diffusion models.
  arXiv  Detail & Related papers  (2023-06-22T10:10:34Z)
• Semi-Implicit Denoising Diffusion Models (SIDDMs) [50.30163684539586]
  Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. We introduce a novel approach that tackles the problem by matching implicit and explicit factors. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps.
  arXiv  Detail & Related papers  (2023-06-21T18:49:22Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces [0.0]
  We develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process. As an example, we introduce Blackout Diffusion'', which learns to produce samples from an empty image instead of from noise.
  arXiv  Detail & Related papers  (2023-05-18T16:24:12Z)
• Star-Shaped Denoising Diffusion Probabilistic Models [5.167803438665587]
  We introduce Star-Shaped DDPM (SSDDPM) Our implementation is available at https://github.com/andreyokhotin/star-shaped.
  arXiv  Detail & Related papers  (2023-02-10T14:16:21Z)
• Denoising Diffusion Gamma Models [91.22679787578438]
  We introduce the Denoising Diffusion Gamma Model (DDGM) and show that noise from Gamma distribution provides improved results for image and speech generation. Our approach preserves the ability to efficiently sample state in the training diffusion process while using Gamma noise.
  arXiv  Detail & Related papers  (2021-10-10T10:46:31Z)
• Non Gaussian Denoising Diffusion Models [91.22679787578438]
  We show that noise from Gamma distribution provides improved results for image and speech generation. We also show that using a mixture of Gaussian noise variables in the diffusion process improves the performance over a diffusion process that is based on a single distribution.
  arXiv  Detail & Related papers  (2021-06-14T16:42:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.