Related papers: Robust Score Matching

Robust Score Matching

URL: http://arxiv.org/abs/2501.05105v1
Date: Thu, 09 Jan 2025 09:46:27 GMT
Title: Robust Score Matching
Authors: Richard Schwank, Andrew McCormack, Mathias Drton,
Abstract summary: We develop a robust score matching procedure that yields consistent parameter estimates in settings where the observed data has been contaminated. A special appeal of the proposed method is that it retains convexity in exponential family models. Support recovery is studied in numerical experiments and on a precipitation dataset.
Score: 1.2835555561822447
License:
Abstract: Proposed in Hyv\"arinen (2005), score matching is a parameter estimation procedure that does not require computation of distributional normalizing constants. In this work we utilize the geometric median of means to develop a robust score matching procedure that yields consistent parameter estimates in settings where the observed data has been contaminated. A special appeal of the proposed method is that it retains convexity in exponential family models. The new method is therefore particularly attractive for non-Gaussian, exponential family graphical models where evaluation of normalizing constants is intractable. Support recovery guarantees for such models when contamination is present are provided. Additionally, support recovery is studied in numerical experiments and on a precipitation dataset. We demonstrate that the proposed robust score matching estimator performs comparably to the standard score matching estimator when no contamination is present but greatly outperforms this estimator in a setting with contamination.

Related papers

Deep Learning Methods for the Noniterative Conditional Expectation G-Formula for Causal Inference from Complex Observational Data [3.0958655016140892]
The g-formula can be used to estimate causal effects of sustained treatment strategies using observational data. Parametric models are subject to model misspecification, which may result in biased causal estimates. We propose a unified deep learning framework for the NICE g-formula estimator.
arXiv Detail & Related papers (2024-10-28T21:00:46Z)
Conformal Approach To Gaussian Process Surrogate Evaluation With Coverage Guarantees [47.22930583160043]
We propose a method for building adaptive cross-conformal prediction intervals. The resulting conformal prediction intervals exhibit a level of adaptivity akin to Bayesian credibility sets. The potential applicability of the method is demonstrated in the context of surrogate modeling of an expensive-to-evaluate simulator of the clogging phenomenon in steam generators of nuclear reactors.
arXiv Detail & Related papers (2024-01-15T14:45:18Z)
Statistical Barriers to Affine-equivariant Estimation [10.077727846124633]
We investigate the quantitative performance of affine-equivariant estimators for robust mean estimation. We find that classical estimators are either quantitatively sub-optimal or lack any quantitative guarantees. We construct a new affine-equivariant estimator which nearly matches our lower bound.
arXiv Detail & Related papers (2023-10-16T18:42:00Z)
A Tale of Sampling and Estimation in Discounted Reinforcement Learning [50.43256303670011]
We present a minimax lower bound on the discounted mean estimation problem. We show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties.
arXiv Detail & Related papers (2023-04-11T09:13:17Z)
The Implicit Delta Method [61.36121543728134]
In this paper, we propose an alternative, the implicit delta method, which works by infinitesimally regularizing the training loss of uncertainty. We show that the change in the evaluation due to regularization is consistent for the variance of the evaluation estimator, even when the infinitesimal change is approximated by a finite difference.
arXiv Detail & Related papers (2022-11-11T19:34:17Z)
Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
arXiv Detail & Related papers (2022-10-03T06:09:01Z)
Near-optimal inference in adaptive linear regression [60.08422051718195]
Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
arXiv Detail & Related papers (2021-07-05T21:05:11Z)
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
arXiv Detail & Related papers (2021-04-11T09:50:24Z)
Tolerance and Prediction Intervals for Non-normal Models [0.0]
A prediction interval covers a future observation from a random process in repeated sampling. A tolerance interval covers a population percentile in repeated sampling and is often based on a pivotal quantity.
arXiv Detail & Related papers (2020-11-23T17:48:09Z)
Nonparametric Score Estimators [49.42469547970041]
Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
arXiv Detail & Related papers (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.