Score Matching for Truncated Density Estimation on a Manifold

• URL: http://arxiv.org/abs/2206.14668v2
• Date: Fri, 12 Apr 2024 14:32:44 GMT
• Title: Score Matching for Truncated Density Estimation on a Manifold
• Authors: Daniel J. Williams, Song Liu,
• Abstract summary: Recent methods propose to use score matching for truncated density estimation. We present a novel extension of truncated score matching to a Riemannian manifold with boundary. In simulated data experiments, our score matching estimator is able to approximate the true parameter values with a low estimation error.
• Score: 6.53626518989653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When observations are truncated, we are limited to an incomplete picture of our dataset. Recent methods propose to use score matching for truncated density estimation, where the access to the intractable normalising constant is not required. We present a novel extension of truncated score matching to a Riemannian manifold with boundary. Applications are presented for the von Mises-Fisher and Kent distributions on a two dimensional sphere in $\mathbb{R}^3$, as well as a real-world application of extreme storm observations in the USA. In simulated data experiments, our score matching estimator is able to approximate the true parameter values with a low estimation error and shows improvements over a naive maximum likelihood estimator.

Related papers

• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection [41.41164637577005]
  Post-hoc out-of-distribution (OOD) detection has garnered intensive attention in reliable machine learning. We propose a novel theoretical framework grounded in Bregman divergence to provide a unified perspective on density-based score design. We show that our proposed textscConjNorm has established a new state-of-the-art in a variety of OOD detection setups.
  arXiv  Detail & Related papers  (2024-02-27T21:02:47Z)
• On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates [5.13323375365494]
  We provide theoretical guarantees for the convergence behaviour of diffusion-based generative models under strongly log-concave data. Our class of functions used for score estimation is made of Lipschitz continuous functions avoiding any Lipschitzness assumption on the score function. This approach yields the best known convergence rate for our sampling algorithm.
  arXiv  Detail & Related papers  (2023-11-22T18:40:45Z)
• Concrete Score Matching: Generalized Score Matching for Discrete Data [109.12439278055213]
  "Concrete score" is a generalization of the (Stein) score for discrete settings. "Concrete Score Matching" is a framework to learn such scores from samples.
  arXiv  Detail & Related papers  (2022-11-02T00:41:37Z)
• Density Estimation with Autoregressive Bayesian Predictives [1.5771347525430772]
  In the context of density estimation, the standard Bayesian approach is to target the posterior predictive. We develop a novel parameterization of the bandwidth using an autoregressive neural network that maps the data into a latent space.
  arXiv  Detail & Related papers  (2022-06-13T20:43:39Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Evaluating State-of-the-Art Classification Models Against Bayes Optimality [106.50867011164584]
  We show that we can compute the exact Bayes error of generative models learned using normalizing flows. We use our approach to conduct a thorough investigation of state-of-the-art classification models.
  arXiv  Detail & Related papers  (2021-06-07T06:21:20Z)
• Distributionally Robust Local Non-parametric Conditional Estimation [22.423052432220235]
  We propose a new distributionally robust estimator that generates non-parametric local estimates. We show that despite being generally intractable, the local estimator can be efficiently found via convex optimization. Experiments with synthetic and MNIST datasets show the competitive performance of this new class of estimators.
  arXiv  Detail & Related papers  (2020-10-12T00:11:17Z)
• Manifold-adaptive dimension estimation revisited [0.0]
  We revisit and improve the manifold-adaptive Farahmand-Szepesv'ari-Audibert dimension estimator. We compute the probability density function of local FSA estimates. We derive the maximum likelihood formula for global intrinsic dimensionality.
  arXiv  Detail & Related papers  (2020-08-07T15:27:26Z)
• Variable Skipping for Autoregressive Range Density Estimation [84.60428050170687]
  We show a technique, variable skipping, for accelerating range density estimation over deep autoregressive models. We show that variable skipping provides 10-100$times$ efficiency improvements when targeting challenging high-quantile error metrics.
  arXiv  Detail & Related papers  (2020-07-10T19:01:40Z)
• TraDE: Transformers for Density Estimation [101.20137732920718]
  TraDE is a self-attention-based architecture for auto-regressive density estimation. We present a suite of tasks such as regression using generated samples, out-of-distribution detection, and robustness to noise in the training data.
  arXiv  Detail & Related papers  (2020-04-06T07:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.