Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

• URL: http://arxiv.org/abs/1912.05737v6
• Date: Thu, 13 Feb 2025 13:23:14 GMT
• Title: Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
• Authors: Badr-Eddine Chérief-Abdellatif, Pierre Alquier,
• Abstract summary: We show that the estimator is robust to both dependence and to the presence of outliers in the dataset. We provide a theoretical study of the gradient descent algorithm used to compute the estimator.
• Score: 7.011897575776511
• License:
• Abstract: Many works in statistics aim at designing a universal estimation procedure, that is, an estimator that would converge to the best approximation of the (unknown) data generating distribution in a model, without any assumption on this distribution. This question is of major interest, in particular because the universality property leads to the robustness of the estimator. In this paper, we tackle the problem of universal estimation using a minimum distance estimator presented in Briol et al. (2019) based on the Maximum Mean Discrepancy. We show that the estimator is robust to both dependence and to the presence of outliers in the dataset. Finally, we provide a theoretical study of the stochastic gradient descent algorithm used to compute the estimator, and we support our findings with numerical simulations. ** The proof of Proposition 4.4 in the published version contains a mistake. The mistake is fixed here (and the bound is actually improved by a factor 2). **

Related papers

• Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances [2.317910166616341]
  In statistical inference, we commonly assume that samples are independent and identically distributed from a probability distribution. In this paper, we explore the application of GAN-like estimators to a general class of statistical models.
  arXiv  Detail & Related papers  (2024-10-28T05:50:47Z)
• Consistent Estimation of a Class of Distances Between Covariance Matrices [7.291687946822539]
  We are interested in the family of distances that can be expressed as sums of traces of functions that are separately applied to each covariance matrix. A statistical analysis of the behavior of this class of distance estimators has also been conducted. We present a central limit theorem that establishes the Gaussianity of these estimators and provides closed form expressions for the corresponding means and variances.
  arXiv  Detail & Related papers  (2024-09-18T07:36:25Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Doubly Robust Inference in Causal Latent Factor Models [12.116813197164047]
  This article introduces a new estimator of average treatment effects under unobserved confounding in modern data-rich environments featuring large numbers of units and outcomes. We derive finite-sample weighting and guarantees, and show that the error of the new estimator converges to a mean-zero Gaussian distribution at a parametric rate.
  arXiv  Detail & Related papers  (2024-02-18T17:13:46Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Robust W-GAN-Based Estimation Under Wasserstein Contamination [8.87135311567798]
  We study several estimation problems under a Wasserstein contamination model and present computationally tractable estimators motivated by generative networks (GANs) Specifically, we analyze properties of Wasserstein GAN-based estimators for adversarial location estimation, covariance matrix estimation, and linear regression. Our proposed estimators are minimax optimal in many scenarios.
  arXiv  Detail & Related papers  (2021-01-20T05:15:16Z)
• Distributionally Robust Parametric Maximum Likelihood Estimation [13.09499764232737]
  We propose a distributionally robust maximum likelihood estimator that minimizes the worst-case expected log-loss uniformly over a parametric nominal distribution. Our novel robust estimator also enjoys statistical consistency and delivers promising empirical results in both regression and classification tasks.
  arXiv  Detail & Related papers  (2020-10-11T19:05:49Z)
• Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator [93.05919133288161]
  We show that the variance of the straight-through variant of the popular Gumbel-Softmax estimator can be reduced through Rao-Blackwellization. This provably reduces the mean squared error. We empirically demonstrate that this leads to variance reduction, faster convergence, and generally improved performance in two unsupervised latent variable models.
  arXiv  Detail & Related papers  (2020-10-09T22:54:38Z)
• $\gamma$-ABC: Outlier-Robust Approximate Bayesian Computation Based on a Robust Divergence Estimator [95.71091446753414]
  We propose to use a nearest-neighbor-based $gamma$-divergence estimator as a data discrepancy measure. Our method achieves significantly higher robustness than existing discrepancy measures.
  arXiv  Detail & Related papers  (2020-06-13T06:09:27Z)
• Nonparametric Estimation of the Fisher Information and Its Applications [82.00720226775964]
  This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. An estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. A new estimator, termed a clipped estimator, is proposed.
  arXiv  Detail & Related papers  (2020-05-07T17:21:56Z)
• Estimating Gradients for Discrete Random Variables by Sampling without Replacement [93.09326095997336]
  We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement. We show that our estimator can be derived as the Rao-Blackwellization of three different estimators.
  arXiv  Detail & Related papers  (2020-02-14T14:15:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.