Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation

• URL: http://arxiv.org/abs/2605.22950v1
• Date: Thu, 21 May 2026 18:25:22 GMT
• Title: Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation
• Abstract summary: Vanilla score matching has shown to be inefficient relative to maximum likelihood estimation for multimodal distributions.<n>We prove statistical guarantees for both estimators, showing that the error bound for the vanilla worsens when the separation between the modes increases.<n>This provides a novel theoretical explanation for the superior behavior of diffusion-based score matching over the vanilla version.
• Score: 7.404568009919419
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Score matching is an alternative to maximum likelihood estimation when the normalizing constant is unknown or too costly to evaluate. However, vanilla score matching has shown to be inefficient relative to maximum likelihood estimation for multimodal distributions with well-separated modes, which are commonly encountered in practical applications. We compare a novel diffusion-based denoising score matching estimator (DDSME) to the vanilla score matching estimator (SME) in this scenario. In particular, we prove statistical guarantees for both estimators, showing that the error bound for the vanilla SME worsens when the separation between the modes increases, which can be avoided in case of the DDSME with suitable hyperparameter tuning. This provides a novel theoretical explanation for the superior behavior of diffusion-based score matching over the vanilla version.

Related papers

• Generalized Score Matching for Parameter Estimation on Convex Domains [13.954816561132942]
  We derive the generalized score matching objective on a convex subset of $mathbbRd$.<n>We show how classical score matching as well as domain-adapted variants for non-negative data arise naturally within the proposed framework.<n>Our derivation sheds new light on the scope and applicability of generalized score matching in various problem settings.
  arXiv  Detail & Related papers  (2026-09-10T13:23:41Z)
• Robust Bayesian Inference for Unnormalized Models with Mixed-Domain Data [2.842028685390757]
  SME-BETEL is a semiparametric framework that combines score matching equations with exponentially tilted empirical likelihood.<n>We show that SME-BETEL credible sets are normalizingally calibrated to the sampling variability of the score matching estimator.<n>This construction enables robust Bayesian inference for mixed-domain doubly-intractable models.
  arXiv  Detail & Related papers  (2026-09-01T18:51:42Z)
• Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference [48.78219918881965]
  We propose a principled framework for deriving task-specific strictly proper scoring rules by matching the local curvature of the downstream error metric.<n>We apply this to the Average Treatment Effect (ATE) estimation, deriving a closed-form loss and its corresponding canonical probability mapping.
  arXiv  Detail & Related papers  (2026-06-02T08:41:25Z)
• Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics [80.05951561886123]
  We leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories.<n>We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis.
  arXiv  Detail & Related papers  (2026-02-27T17:27:55Z)
• Variance-Reduced Diffusion Sampling via Target Score Identity [0.0]
  We study variance reduction for score estimation and diffusion-based sampling in settings where the clean (target) score is available or can be approximated.<n>We develop a plug-and-play nonparametric self-normalized importance sampling estimator compatible with standard reverse-time solvers.<n> Experiments on synthetic targets and PDE-governed inverse problems demonstrate improved sample quality for a fixed simulation budget.
  arXiv  Detail & Related papers  (2026-01-04T16:46:31Z)
• Implicit score matching meets denoising score matching: improved rates of convergence and log-density Hessian estimation [5.773269033551628]
  We study the problem of estimating the score function using both implicit score matching and denoising score matching.<n>We prove that implicit score matching is able not only to adapt to the intrinsic dimension, but also to achieve the same rates of convergence as denoising score matching.
  arXiv  Detail & Related papers  (2025-12-30T17:39:48Z)
• ScoreMatchingRiesz: Auto-DML with Infinitesimal Classification [6.44705221140412]
  The Riesz representer is a key component in machine learning for constructing $sqrtn$-consistent and efficient estimators.<n>We extend score-matching-based DRE methods to Riesz representer estimation.
  arXiv  Detail & Related papers  (2025-12-23T17:14:14Z)
• Control Variate Score Matching for Diffusion Models [34.30408848157335]
  We introduce the Control Variate Score Identity (CVSI), deriving an optimal, time-dependent control coefficient that theoretically guarantees variance minimization across the entire noise spectrum.<n>We demonstrate that CVSI serves as a robust, low-variance plug-in estimator that significantly enhances sample efficiency in both data-free sampler learning and inference-time diffusion sampling.
  arXiv  Detail & Related papers  (2025-12-23T02:55:14Z)
• Lower Bounds on the MMSE of Adversarially Inferring Sensitive Features [4.8998185508205765]
  We propose an adversarial evaluation framework for sensitive feature inference based on a finite sample size and linear predictive models.<n>Our approach establishes theoretical lower bounds on the true MMSE of inferring sensitive features from noisy observations of other correlated features.
  arXiv  Detail & Related papers  (2025-05-13T22:39:24Z)
• In-Context Parametric Inference: Point or Distribution Estimators? [66.22308335324239]
  We show that amortized point estimators generally outperform posterior inference, though the latter remain competitive in some low-dimensional problems.<n>Our experiments indicate that amortized point estimators generally outperform posterior inference, though the latter remain competitive in some low-dimensional problems.
  arXiv  Detail & Related papers  (2025-02-17T10:00:24Z)
• Diffusion models for probabilistic programming [56.47577824219207]
  Diffusion Model Variational Inference (DMVI) is a novel method for automated approximate inference in probabilistic programming languages (PPLs) DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model.
  arXiv  Detail & Related papers  (2023-11-01T12:17:05Z)
• Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
  We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
  arXiv  Detail & Related papers  (2022-10-03T06:09:01Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models [80.22609163316459]
  We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost.
  arXiv  Detail & Related papers  (2020-04-01T11:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.