Pareto Smoothed Importance Sampling
- URL: http://arxiv.org/abs/1507.02646v9
- Date: Wed, 13 Mar 2024 15:54:02 GMT
- Title: Pareto Smoothed Importance Sampling
- Authors: Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao, Jonah Gabry,
- Abstract summary: Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution.
This routinely occurs when there are aspects of the target distribution that are not well captured by the approximating distribution.
We present a new method for stabilizing importance weights using a generalized Pareto distribution fit to the upper tail of the distribution of the simulated importance ratios.
- Score: 8.705872384531318
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail. This routinely occurs when there are aspects of the target distribution that are not well captured by the approximating distribution, in which case more stable estimates can be obtained by modifying extreme importance ratios. We present a new method for stabilizing importance weights using a generalized Pareto distribution fit to the upper tail of the distribution of the simulated importance ratios. The method, which empirically performs better than existing methods for stabilizing importance sampling estimates, includes stabilized effective sample size estimates, Monte Carlo error estimates, and convergence diagnostics. The presented Pareto $\hat{k}$ finite sample convergence rate diagnostic is useful for any Monte Carlo estimator.
Related papers
- On Tail Decay Rate Estimation of Loss Function Distributions [5.33024001730262]
We develop a novel theory for estimating the tails of marginal distributions.
We show that under some regularity conditions, the shape parameter of the marginal distribution is the maximum tail shape parameter of the family of conditional distributions.
arXiv Detail & Related papers (2023-06-05T11:58:25Z) - Concentration inequalities for leave-one-out cross validation [1.90365714903665]
We show that estimator stability is enough to show that leave-one-out cross validation is a sound procedure.
We obtain our results by relying on random variables with distribution satisfying the logarithmic Sobolev inequality.
arXiv Detail & Related papers (2022-11-04T14:08:53Z) - A Bayesian Semiparametric Method For Estimating Causal Quantile Effects [1.1118668841431563]
We propose a semiparametric conditional distribution regression model that allows inference on any functionals of counterfactual distributions.
We show via simulations that the use of double balancing score for confounding adjustment improves performance over adjusting for any single score alone.
We apply the proposed method to the North Carolina birth weight dataset to analyze the effect of maternal smoking on infant's birth weight.
arXiv Detail & Related papers (2022-11-03T05:15:18Z) - Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated.
We formalize these results both in the sample regime and in the finite regime.
arXiv Detail & Related papers (2022-10-03T06:09:01Z) - RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and
Out Distribution Robustness [94.69774317059122]
We show that the effectiveness of the well celebrated Mixup can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss.
This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup.
arXiv Detail & Related papers (2022-06-29T09:44:33Z) - Adaptive Importance Sampling meets Mirror Descent: a Bias-variance
tradeoff [7.538482310185135]
A major drawback of adaptive importance sampling is the large variance of the weights.
This paper investigates a regularization strategy whose basic principle is to raise the importance weights at a certain power.
arXiv Detail & Related papers (2021-10-29T07:45:24Z) - Variational Refinement for Importance Sampling Using the Forward
Kullback-Leibler Divergence [77.06203118175335]
Variational Inference (VI) is a popular alternative to exact sampling in Bayesian inference.
Importance sampling (IS) is often used to fine-tune and de-bias the estimates of approximate Bayesian inference procedures.
We propose a novel combination of optimization and sampling techniques for approximate Bayesian inference.
arXiv Detail & Related papers (2021-06-30T11:00:24Z) - Batch Stationary Distribution Estimation [98.18201132095066]
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions.
We propose a consistent estimator that is based on recovering a correction ratio function over the given data.
arXiv Detail & Related papers (2020-03-02T09:10:01Z) - Efficiently Sampling Functions from Gaussian Process Posteriors [76.94808614373609]
We propose an easy-to-use and general-purpose approach for fast posterior sampling.
We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
arXiv Detail & Related papers (2020-02-21T14:03:16Z) - Distributionally Robust Bayesian Quadrature Optimization [60.383252534861136]
We study BQO under distributional uncertainty in which the underlying probability distribution is unknown except for a limited set of its i.i.d. samples.
A standard BQO approach maximizes the Monte Carlo estimate of the true expected objective given the fixed sample set.
We propose a novel posterior sampling based algorithm, namely distributionally robust BQO (DRBQO) for this purpose.
arXiv Detail & Related papers (2020-01-19T12:00:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.