Sampling-Based Accuracy Testing of Posterior Estimators for General
Inference
- URL: http://arxiv.org/abs/2302.03026v2
- Date: Fri, 2 Jun 2023 20:28:38 GMT
- Title: Sampling-Based Accuracy Testing of Posterior Estimators for General
Inference
- Authors: Pablo Lemos, Adam Coogan, Yashar Hezaveh, Laurence Perreault-Levasseur
- Abstract summary: Generative models can be used as an alternative to Markov Chain Monte Carlo methods for posterior inference.
We introduce Tests of Accuracy with Random Points' (TARP) coverage testing as a method to estimate coverage probabilities of generative posterior estimators.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Parameter inference, i.e. inferring the posterior distribution of the
parameters of a statistical model given some data, is a central problem to many
scientific disciplines. Generative models can be used as an alternative to
Markov Chain Monte Carlo methods for conducting posterior inference, both in
likelihood-based and simulation-based problems. However, assessing the accuracy
of posteriors encoded in generative models is not straightforward. In this
paper, we introduce `Tests of Accuracy with Random Points' (TARP) coverage
testing as a method to estimate coverage probabilities of generative posterior
estimators. Our method differs from previously-existing coverage-based methods,
which require posterior evaluations. We prove that our approach is necessary
and sufficient to show that a posterior estimator is accurate. We demonstrate
the method on a variety of synthetic examples, and show that TARP can be used
to test the results of posterior inference analyses in high-dimensional spaces.
We also show that our method can detect inaccurate inferences in cases where
existing methods fail.
Related papers
- DistPred: A Distribution-Free Probabilistic Inference Method for Regression and Forecasting [14.390842560217743]
We propose a novel approach called DistPred for regression and forecasting tasks.
We transform proper scoring rules that measure the discrepancy between the predicted distribution and the target distribution into a differentiable discrete form.
This allows the model to sample numerous samples in a single forward pass to estimate the potential distribution of the response variable.
arXiv Detail & Related papers (2024-06-17T10:33:00Z) - Reducing the cost of posterior sampling in linear inverse problems via task-dependent score learning [5.340736751238338]
We show that the evaluation of the forward mapping can be entirely bypassed during posterior sample generation.
We prove that this observation generalizes to the framework of infinite-dimensional diffusion models introduced recently.
arXiv Detail & Related papers (2024-05-24T15:33:27Z) - A variational neural Bayes framework for inference on intractable posterior distributions [1.0801976288811024]
Posterior distributions of model parameters are efficiently obtained by feeding observed data into a trained neural network.
We show theoretically that our posteriors converge to the true posteriors in Kullback-Leibler divergence.
arXiv Detail & Related papers (2024-04-16T20:40:15Z) - Calibrating Neural Simulation-Based Inference with Differentiable
Coverage Probability [50.44439018155837]
We propose to include a calibration term directly into the training objective of the neural model.
By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation.
It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference.
arXiv Detail & Related papers (2023-10-20T10:20:45Z) - Selective Nonparametric Regression via Testing [54.20569354303575]
We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point.
Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
arXiv Detail & Related papers (2023-09-28T13:04:11Z) - Compositional Score Modeling for Simulation-based Inference [28.422049267537965]
We introduce a new method based on conditional score modeling that enjoys the benefits of both approaches.
Our approach is sample-efficient, can naturally aggregate multiple observations at inference time, and avoids the drawbacks of standard inference methods.
arXiv Detail & Related papers (2022-09-28T17:08:31Z) - A Provably Efficient Model-Free Posterior Sampling Method for Episodic
Reinforcement Learning [50.910152564914405]
Existing posterior sampling methods for reinforcement learning are limited by being model-based or lack worst-case theoretical guarantees beyond linear MDPs.
This paper proposes a new model-free formulation of posterior sampling that applies to more general episodic reinforcement learning problems with theoretical guarantees.
arXiv Detail & Related papers (2022-08-23T12:21:01Z) - Scalable Marginal Likelihood Estimation for Model Selection in Deep
Learning [78.83598532168256]
Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties.
Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
arXiv Detail & Related papers (2021-04-11T09:50:24Z) - Bayesian Imaging With Data-Driven Priors Encoded by Neural Networks:
Theory, Methods, and Algorithms [2.266704469122763]
This paper proposes a new methodology for performing Bayesian inference in imaging inverse problems where the prior knowledge is available in the form of training data.
We establish the existence and well-posedness of the associated posterior moments under easily verifiable conditions.
A model accuracy analysis suggests that the Bayesian probability probabilities reported by the data-driven models are also remarkably accurate under a frequentist definition.
arXiv Detail & Related papers (2021-03-18T11:34:08Z) - Tracking disease outbreaks from sparse data with Bayesian inference [55.82986443159948]
The COVID-19 pandemic provides new motivation for estimating the empirical rate of transmission during an outbreak.
Standard methods struggle to accommodate the partial observability and sparse data common at finer scales.
We propose a Bayesian framework which accommodates partial observability in a principled manner.
arXiv Detail & Related papers (2020-09-12T20:37:33Z) - Machine learning for causal inference: on the use of cross-fit
estimators [77.34726150561087]
Doubly-robust cross-fit estimators have been proposed to yield better statistical properties.
We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE)
When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
arXiv Detail & Related papers (2020-04-21T23:09:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.