Accelerated Diffusion Models via Speculative Sampling

• URL: http://arxiv.org/abs/2501.05370v1
• Date: Thu, 09 Jan 2025 16:50:16 GMT
• Title: Accelerated Diffusion Models via Speculative Sampling
• Authors: Valentin De Bortoli, Alexandre Galashov, Arthur Gretton, Arnaud Doucet,
• Abstract summary: Speculative sampling is a popular technique for accelerating inference in Large Language Models. We extend speculative sampling to diffusion models, which generate samples via continuous, vector-valued Markov chains. We propose various drafting strategies, including a simple and effective approach that does not require training a draft model.
• Score: 89.43940130493233
• License:
• Abstract: Speculative sampling is a popular technique for accelerating inference in Large Language Models by generating candidate tokens using a fast draft model and accepting or rejecting them based on the target model's distribution. While speculative sampling was previously limited to discrete sequences, we extend it to diffusion models, which generate samples via continuous, vector-valued Markov chains. In this context, the target model is a high-quality but computationally expensive diffusion model. We propose various drafting strategies, including a simple and effective approach that does not require training a draft model and is applicable out of the box to any diffusion model. Our experiments demonstrate significant generation speedup on various diffusion models, halving the number of function evaluations, while generating exact samples from the target model.

Related papers

• Generative Modeling with Bayesian Sample Inference [50.07758840675341]
  We derive a novel generative model from the simple act of Gaussian posterior inference. Treating the generated sample as an unknown variable to infer lets us formulate the sampling process in the language of Bayesian probability. Our model uses a sequence of prediction and posterior update steps to narrow down the unknown sample from a broad initial belief.
  arXiv  Detail & Related papers  (2025-02-11T14:27:10Z)
• Generative Modeling with Diffusion [0.0]
  We introduce the diffusion model as a method to generate new samples. We will define the noising and denoising processes, then introduce algorithms to train and generate with a diffusion model.
  arXiv  Detail & Related papers  (2024-12-14T20:04:46Z)
• Random Walks with Tweedie: A Unified Framework for Diffusion Models [11.161487364062667]
  We present a simple template for designing generative diffusion model algorithms based on an interpretation of diffusion sampling as a sequence of random walks. We show that several existing diffusion models correspond to particular choices within this template and demonstrate that other, more straightforward algorithmic choices lead to effective diffusion models.
  arXiv  Detail & Related papers  (2024-11-27T19:13:20Z)
• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step. Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• Provable Statistical Rates for Consistency Diffusion Models [87.28777947976573]
  Despite the state-of-the-art performance, diffusion models are known for their slow sample generation due to the extensive number of steps involved. This paper contributes towards the first statistical theory for consistency models, formulating their training as a distribution discrepancy minimization problem.
  arXiv  Detail & Related papers  (2024-06-23T20:34:18Z)
• Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time [49.598085130313514]
  We propose discrete non-Markov diffusion models (DNDM), which naturally induce the predetermined transition time set. This enables a training-free sampling algorithm that significantly reduces the number of function evaluations. We study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes.
  arXiv  Detail & Related papers  (2023-12-14T18:14:11Z)
• Fast Inference in Denoising Diffusion Models via MMD Finetuning [23.779985842891705]
  We present MMD-DDM, a novel method for fast sampling of diffusion models. Our approach is based on the idea of using the Maximum Mean Discrepancy (MMD) to finetune the learned distribution with a given budget of timesteps. Our findings show that the proposed method is able to produce high-quality samples in a fraction of the time required by widely-used diffusion models.
  arXiv  Detail & Related papers  (2023-01-19T09:48:07Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.