Nonparametric Estimation of the Fisher Information and Its Applications

• URL: http://arxiv.org/abs/2005.03622v1
• Date: Thu, 7 May 2020 17:21:56 GMT
• Title: Nonparametric Estimation of the Fisher Information and Its Applications
• Authors: Wei Cao, Alex Dytso, Michael Fau{\ss}, H. Vincent Poor, and Gang Feng
• Abstract summary: This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. An estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. A new estimator, termed a clipped estimator, is proposed.
• Score: 82.00720226775964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed a clipped estimator, is proposed. Superior upper bounds on the rates of convergence can be shown for the new estimator compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown's identity, which relates the Fisher information and the minimum mean squared error (MMSE) in Gaussian noise, two corresponding consistent estimators for the MMSE are proposed. Simulation examples for the Bhattacharya estimator and the clipped estimator as well as the MMSE estimators are presented. The examples demonstrate that the clipped estimator can significantly reduce the required sample size to guarantee a specific confidence interval compared to the Bhattacharya estimator.

Related papers

• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Statistical Barriers to Affine-equivariant Estimation [10.077727846124633]
  We investigate the quantitative performance of affine-equivariant estimators for robust mean estimation. We find that classical estimators are either quantitatively sub-optimal or lack any quantitative guarantees. We construct a new affine-equivariant estimator which nearly matches our lower bound.
  arXiv  Detail & Related papers  (2023-10-16T18:42:00Z)
• Leveraging Variational Autoencoders for Parameterized MMSE Estimation [10.141454378473972]
  We propose a variational autoencoder-based framework for parameterizing a conditional linear minimum mean squared error estimator. The derived estimator is shown to approximate the minimum mean squared error estimator by utilizing the variational autoencoder as a generative prior for the estimation problem. We conduct a rigorous analysis by bounding the difference between the proposed and the minimum mean squared error estimator.
  arXiv  Detail & Related papers  (2023-07-11T15:41:34Z)
• A Tale of Sampling and Estimation in Discounted Reinforcement Learning [50.43256303670011]
  We present a minimax lower bound on the discounted mean estimation problem. We show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties.
  arXiv  Detail & Related papers  (2023-04-11T09:13:17Z)
• Robust W-GAN-Based Estimation Under Wasserstein Contamination [8.87135311567798]
  We study several estimation problems under a Wasserstein contamination model and present computationally tractable estimators motivated by generative networks (GANs) Specifically, we analyze properties of Wasserstein GAN-based estimators for adversarial location estimation, covariance matrix estimation, and linear regression. Our proposed estimators are minimax optimal in many scenarios.
  arXiv  Detail & Related papers  (2021-01-20T05:15:16Z)
• Distributionally Robust Parametric Maximum Likelihood Estimation [13.09499764232737]
  We propose a distributionally robust maximum likelihood estimator that minimizes the worst-case expected log-loss uniformly over a parametric nominal distribution. Our novel robust estimator also enjoys statistical consistency and delivers promising empirical results in both regression and classification tasks.
  arXiv  Detail & Related papers  (2020-10-11T19:05:49Z)
• Learning Minimax Estimators via Online Learning [55.92459567732491]
  We consider the problem of designing minimax estimators for estimating parameters of a probability distribution. We construct an algorithm for finding a mixed-case Nash equilibrium.
  arXiv  Detail & Related papers  (2020-06-19T22:49:42Z)
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models [80.22609163316459]
  We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost.
  arXiv  Detail & Related papers  (2020-04-01T11:49:30Z)
• Estimating Gradients for Discrete Random Variables by Sampling without Replacement [93.09326095997336]
  We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement. We show that our estimator can be derived as the Rao-Blackwellization of three different estimators.
  arXiv  Detail & Related papers  (2020-02-14T14:15:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.