Optimal score function estimation via derivatives constraints

• URL: http://arxiv.org/abs/2606.19084v1
• Date: Wed, 17 Jun 2026 13:55:20 GMT
• Title: Optimal score function estimation via derivatives constraints
• Abstract summary: We show that constraining the hypothesis space to a Sobolev ball is sufficient to prevent overfitting and obtaining minimax estimation rates.<n>We then consider the problem of score function estimation in the context of score-based generative modeling.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of score function estimation via empirical risk minimization. We first start with the question of inferring the score function of a probability measure $μ$ with density on the flat torus from a sample of distribution $μ$. We show that constraining the hypothesis space to a Sobolev ball is sufficient to prevent overfitting and obtaining minimax estimation rates. We then consider the problem of score function estimation in the context of score-based generative modeling. Again, under a conjecture tying the score estimation rates to the quality of the output of a score-based generative model, we obtain minimax rates for such an approach using score function estimators obtained by constraining the hypothesis class to a Sobolev ball.

Related papers

• Generalized Score Matching for Parameter Estimation on Convex Domains [13.954816561132942]
  We derive the generalized score matching objective on a convex subset of $mathbbRd$.<n>We show how classical score matching as well as domain-adapted variants for non-negative data arise naturally within the proposed framework.<n>Our derivation sheds new light on the scope and applicability of generalized score matching in various problem settings.
  arXiv  Detail & Related papers  (2026-09-10T13:23:41Z)
• Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees [8.437829850850358]
  We study score-based diffusion models when the target distribution is heavy-tailed.<n>We derive sharp minimax rates for score estimation, revealing a dichotomy.<n>In total variation, the distribution generated at the minimax optimal rate $n-/(2+d)$, and at a $$-dependent rate under exponential tails.
  arXiv  Detail & Related papers  (2026-01-10T23:05:25Z)
• Learning the score under shape constraints [7.005582630391827]
  We study the minimax risk of score estimation with respect to squared $L2(P_0)$-loss.<n>We define subclasses of log-concave densities that capture two fundamental aspects of the estimation problem.<n>We show that the minimax risk over this latter class is of order $L2/(2+1)n-/(2+1)$ up to poly-logarithmic factors.
  arXiv  Detail & Related papers  (2025-12-16T17:39:54Z)
• DDPM Score Matching and Distribution Learning [24.341062891949953]
  Score estimation is the backbone of score-based generative models (SGMs)<n>This paper introduces a framework that reduces score estimation to tasks of parameter and density estimation.<n>We provide minimax rates for density estimation over H" classes and a quasi-polynomial PAC density estimation algorithm.
  arXiv  Detail & Related papers  (2025-04-07T15:07:19Z)
• Optimal score estimation via empirical Bayes smoothing [13.685846094715364]
  We study the problem of estimating the score function of an unknown probability distribution $rho*$ from $n$ independent and identically distributed observations in $d$ dimensions. We show that a regularized score estimator based on a Gaussian kernel attains this rate, shown optimal by a matching minimax lower bound.
  arXiv  Detail & Related papers  (2024-02-12T16:17:40Z)
• Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling [82.36856860383291]
  We demonstrate that accurate estimation of the score function is achievable by training a standard deep ReLU neural network. We establish bounds on the error rate of recovering causal relationships using the score-matching-based causal discovery method.
  arXiv  Detail & Related papers  (2023-10-27T13:09:56Z)
• A Tale of Sampling and Estimation in Discounted Reinforcement Learning [50.43256303670011]
  We present a minimax lower bound on the discounted mean estimation problem. We show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties.
  arXiv  Detail & Related papers  (2023-04-11T09:13:17Z)
• Estimating the minimizer and the minimum value of a regression function under passive design [72.85024381807466]
  We propose a new method for estimating the minimizer $boldsymbolx*$ and the minimum value $f*$ of a smooth and strongly convex regression function $f$. We derive non-asymptotic upper bounds for the quadratic risk and optimization error of $boldsymbolz_n$, and for the risk of estimating $f*$.
  arXiv  Detail & Related papers  (2022-11-29T18:38:40Z)
• Learning Minimax Estimators via Online Learning [55.92459567732491]
  We consider the problem of designing minimax estimators for estimating parameters of a probability distribution. We construct an algorithm for finding a mixed-case Nash equilibrium.
  arXiv  Detail & Related papers  (2020-06-19T22:49:42Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)
• Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation [49.502277468627035]
  This paper studies the statistical theory of batch data reinforcement learning with function approximation. Consider the off-policy evaluation problem, which is to estimate the cumulative value of a new target policy from logged history.
  arXiv  Detail & Related papers  (2020-02-21T19:20:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.