Variance-Reduced Diffusion Sampling via Target Score Identity

• URL: http://arxiv.org/abs/2601.01594v2
• Date: Thu, 08 Jan 2026 21:50:23 GMT
• Title: Variance-Reduced Diffusion Sampling via Target Score Identity
• Authors: Alois Duston, Tan Bui-Thanh,
• Abstract summary: We study variance reduction for score estimation and diffusion-based sampling in settings where the clean (target) score is available or can be approximated.<n>We develop a plug-and-play nonparametric self-normalized importance sampling estimator compatible with standard reverse-time solvers.<n> Experiments on synthetic targets and PDE-governed inverse problems demonstrate improved sample quality for a fixed simulation budget.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study variance reduction for score estimation and diffusion-based sampling in settings where the clean (target) score is available or can be approximated. Starting from the Target Score Identity (TSI), which expresses the noisy marginal score as a conditional expectation of the target score under the forward diffusion, we develop: (i) a plug-and-play nonparametric self-normalized importance sampling estimator compatible with standard reverse-time solvers, (ii) a variance-minimizing \emph{state- and time-dependent} blending rule between Tweedie-type and TSI estimators together with an anti-correlation analysis, (iii) a data-only extension based on locally fitted proxy scores, and (iv) a likelihood-tilting extension to Bayesian inverse problems. We also propose a \emph{Critic--Gate} distillation scheme that amortizes the state-dependent blending coefficient into a neural gate. Experiments on synthetic targets and PDE-governed inverse problems demonstrate improved sample quality for a fixed simulation budget.

Related papers

• Sharp Convergence Rates for Masked Diffusion Models [53.117058231393834]
  We develop a total-variation based analysis for the Euler method that overcomes limitations.<n>Our results relax assumptions on score estimation, improve parameter dependencies, and establish convergence guarantees.<n>Overall, our analysis introduces a direct TV-based error decomposition along the CTMC trajectory and a decoupling-based path-wise analysis for FHS.
  arXiv  Detail & Related papers  (2026-02-26T00:47:51Z)
• Inference-Time Alignment for Diffusion Models via Doob's Matching [16.416975860645724]
  Inference-time alignment for diffusion models aims to adapt a pre-trained diffusion model toward a target distribution without retraining the base score network.<n>We introduce Doob's matching, a novel framework for guidance estimation grounded in Doob's $h$-transform.<n>We prove non-asymptotic convergence guarantees for the generated distributions in the 2-Wasserstein distance.
  arXiv  Detail & Related papers  (2026-01-10T10:28:06Z)
• Control Variate Score Matching for Diffusion Models [34.30408848157335]
  We introduce the Control Variate Score Identity (CVSI), deriving an optimal, time-dependent control coefficient that theoretically guarantees variance minimization across the entire noise spectrum.<n>We demonstrate that CVSI serves as a robust, low-variance plug-in estimator that significantly enhances sample efficiency in both data-free sampler learning and inference-time diffusion sampling.
  arXiv  Detail & Related papers  (2025-12-23T02:55:14Z)
• Likelihood Matching for Diffusion Models [2.17741936620649]
  We propose a Likelihood Matching approach for training diffusion models.<n>A quasi-likelihood is considered to approximate each reverse transition density by a Gaussian distribution.<n>A sampler is introduced to facilitate computation that leverages on both the estimated score and Hessian information.
  arXiv  Detail & Related papers  (2025-08-05T16:51:29Z)
• The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis [0.0]
  We study the effect of divergenceity on the generation process through bounds on the Kullback-Leibler (KL)<n>Our main results apply to linear forward SDEs with additive noise and Lipschitz-continuous score functions.
  arXiv  Detail & Related papers  (2025-06-13T01:01:07Z)
• Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distance [0.0]
  We present a novel framework for analyzing $mathcalW$-convergence in Generative Models (SGMs)<n>We show that weak log-concavity of the data distribution evolves into log-concavity over time.<n>Our approach circumvents the need for stringent regularity conditions on the score function and its estimators.
  arXiv  Detail & Related papers  (2025-01-04T14:33:27Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods [59.779795063072655]
  Chain-of-Thought (CoT) prompting and its variants have gained popularity as effective methods for solving multi-step reasoning problems. We analyze CoT prompting from a statistical estimation perspective, providing a comprehensive characterization of its sample complexity.
  arXiv  Detail & Related papers  (2024-08-25T04:07:18Z)
• Prognostic Covariate Adjustment for Logistic Regression in Randomized Controlled Trials [1.5020330976600735]
  We show that prognostic score adjustment can increase the power of the Wald test for the conditional odds ratio under a fixed sample size. We utilize g-computation to expand the scope of prognostic score adjustment to inferences on the marginal risk difference, relative risk, and odds ratio estimands.
  arXiv  Detail & Related papers  (2024-02-29T06:53:16Z)
• An analysis of the noise schedule for score-based generative models [7.180235086275926]
  Score-based generative models (SGMs) aim at estimating a target data distribution by learning score functions using only noise-perturbed samples from the target.<n>Recent literature has focused extensively on assessing the error between the target and estimated distributions, gauging the generative quality through the Kullback-Leibler (KL) divergence and Wasserstein distances.<n>We establish an upper bound for the KL divergence between the target and the estimated distributions, explicitly depending on any time-dependent noise schedule.
  arXiv  Detail & Related papers  (2024-02-07T08:24:35Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation [72.19198763459448]
  We learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise. These perturbed data densities are linked together by the Fokker-Planck equation (FPE), a partial differential equation (PDE) governing the spatial-temporal evolution of a density. We derive a corresponding equation called the score FPE that characterizes the noise-conditional scores of the perturbed data densities.
  arXiv  Detail & Related papers  (2022-10-09T16:27:25Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• Comparing Probability Distributions with Conditional Transport [63.11403041984197]
  We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
  arXiv  Detail & Related papers  (2020-12-28T05:14:22Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)
• Estimating Gradients for Discrete Random Variables by Sampling without Replacement [93.09326095997336]
  We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement. We show that our estimator can be derived as the Rao-Blackwellization of three different estimators.
  arXiv  Detail & Related papers  (2020-02-14T14:15:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.