Cram\'er-Rao Lower Bounds Arising from Generalized Csisz\'ar Divergences
- URL: http://arxiv.org/abs/2001.04769v2
- Date: Sun, 24 May 2020 05:24:23 GMT
- Title: Cram\'er-Rao Lower Bounds Arising from Generalized Csisz\'ar Divergences
- Authors: M. Ashok Kumar and Kumar Vijay Mishra
- Abstract summary: We study the geometry of probability distributions with respect to a generalized family of Csisz'ar $f$-divergences.
We show that these formulations lead us to find unbiased and efficient estimators for the escort model.
- Score: 17.746238062801293
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We study the geometry of probability distributions with respect to a
generalized family of Csisz\'ar $f$-divergences. A member of this family is the
relative $\alpha$-entropy which is also a R\'enyi analog of relative entropy in
information theory and known as logarithmic or projective power divergence in
statistics. We apply Eguchi's theory to derive the Fisher information metric
and the dual affine connections arising from these generalized divergence
functions. This enables us to arrive at a more widely applicable version of the
Cram\'{e}r-Rao inequality, which provides a lower bound for the variance of an
estimator for an escort of the underlying parametric probability distribution.
We then extend the Amari-Nagaoka's dually flat structure of the exponential and
mixer models to other distributions with respect to the aforementioned
generalized metric. We show that these formulations lead us to find unbiased
and efficient estimators for the escort model. Finally, we compare our work
with prior results on generalized Cram\'er-Rao inequalities that were derived
from non-information-geometric frameworks.
Related papers
- von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
We explore a family of expressive and interpretable distributions over circle-valued random functions.
The resulting probability model has connections with continuous spin models in statistical physics.
For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
arXiv Detail & Related papers (2024-06-19T01:57:21Z) - Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold.
It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
arXiv Detail & Related papers (2023-11-08T15:17:13Z) - Conformal inference for regression on Riemannian Manifolds [49.7719149179179]
We investigate prediction sets for regression scenarios when the response variable, denoted by $Y$, resides in a manifold, and the covariable, denoted by X, lies in Euclidean space.
We prove the almost sure convergence of the empirical version of these regions on the manifold to their population counterparts.
arXiv Detail & Related papers (2023-10-12T10:56:25Z) - Kernel-based off-policy estimation without overlap: Instance optimality
beyond semiparametric efficiency [53.90687548731265]
We study optimal procedures for estimating a linear functional based on observational data.
For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
arXiv Detail & Related papers (2023-01-16T02:57:37Z) - On Best-Arm Identification with a Fixed Budget in Non-Parametric
Multi-Armed Bandits [0.0]
We consider general, possibly non-parametric, models D for distributions over the arms.
We propose upper bounds on the average log-probability of misidentifying the optimal arm based on information-theoretic quantities.
arXiv Detail & Related papers (2022-09-30T10:55:40Z) - Distribution Regression with Sliced Wasserstein Kernels [45.916342378789174]
We propose the first OT-based estimator for distribution regression.
We study the theoretical properties of a kernel ridge regression estimator based on such representation.
arXiv Detail & Related papers (2022-02-08T15:21:56Z) - Robust Estimation for Nonparametric Families via Generative Adversarial
Networks [92.64483100338724]
We provide a framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems.
Our work extend these to robust mean estimation, second moment estimation, and robust linear regression.
In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance.
arXiv Detail & Related papers (2022-02-02T20:11:33Z) - Hybrid and Generalized Bayesian Cram\'{e}r-Rao Inequalities via
Information Geometry [15.33401602207049]
This chapter summarizes the recent results which extend this framework to more general Cram'er-Rao inequalities.
We apply Eguchi's theory to a generalized form of Czsisz'ar $f$-divergence.
arXiv Detail & Related papers (2021-04-02T14:21:49Z) - Generalized Sliced Distances for Probability Distributions [47.543990188697734]
We introduce a broad family of probability metrics, coined as Generalized Sliced Probability Metrics (GSPMs)
GSPMs are rooted in the generalized Radon transform and come with a unique geometric interpretation.
We consider GSPM-based gradient flows for generative modeling applications and show that under mild assumptions, the gradient flow converges to the global optimum.
arXiv Detail & Related papers (2020-02-28T04:18:00Z) - Generalized Bayesian Cram\'{e}r-Rao Inequality via Information Geometry
of Relative $\alpha$-Entropy [17.746238062801293]
relative $alpha$-entropy is the R'enyi analog of relative entropy.
Recent information geometric investigations on this quantity have enabled the generalization of the Cram'er-Rao inequality.
We show that in the limiting case when the entropy order approaches unity, this framework reduces to the conventional Bayesian Cram'er-Rao inequality.
arXiv Detail & Related papers (2020-02-11T23:38:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.