Deep Bayes Factors
- URL: http://arxiv.org/abs/2312.05411v2
- Date: Wed, 12 Jun 2024 11:21:31 GMT
- Title: Deep Bayes Factors
- Authors: Jungeum Kim, Veronika Rockova,
- Abstract summary: We propose a deep learning estimator of the Bayes factor based on simulated data from two competing models.
Our estimator is devoid of summary statistics and obviates some of the difficulties with ABC model choice.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The is no other model or hypothesis verification tool in Bayesian statistics that is as widely used as the Bayes factor. We focus on generative models that are likelihood-free and, therefore, render the computation of Bayes factors (marginal likelihood ratios) far from obvious. We propose a deep learning estimator of the Bayes factor based on simulated data from two competing models using the likelihood ratio trick. This estimator is devoid of summary statistics and obviates some of the difficulties with ABC model choice. We establish sufficient conditions for consistency of our Deep Bayes Factor estimator as well as its consistency as a model selection tool. We investigate the performance of our estimator on various examples using a wide range of quality metrics related to estimation and model decision accuracy. After training, our deep learning approach enables rapid evaluations of the Bayes factor estimator at any fictional data arriving from either hypothesized model, not just the observed data $Y_0$. This allows us to inspect entire Bayes factor distributions under the two models and to quantify the relative location of the Bayes factor evaluated at $Y_0$ in light of these distributions. Such tail area evaluations are not possible for Bayes factor estimators tailored to $Y_0$. We find the performance of our Deep Bayes Factors competitive with existing MCMC techniques that require the knowledge of the likelihood function. We also consider variants for posterior or intrinsic Bayes factors estimation. We demonstrate the usefulness of our approach on a relatively high-dimensional real data example about determining cognitive biases.
Related papers
- BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models [52.46248487458641]
We propose a Bayesian inference framework called BIRD for large language models.
BIRD provides controllable and interpretable probability estimation for model decisions.
Experiments show that BIRD produces probability estimations that align with human judgments over 65% of the time.
arXiv Detail & Related papers (2024-04-18T20:17:23Z) - Efficient Shapley Values Estimation by Amortization for Text
Classification [66.7725354593271]
We develop an amortized model that directly predicts each input feature's Shapley Value without additional model evaluations.
Experimental results on two text classification datasets demonstrate that our amortized model estimates Shapley Values accurately with up to 60 times speedup.
arXiv Detail & Related papers (2023-05-31T16:19:13Z) - Evidence Networks: simple losses for fast, amortized, neural Bayesian
model comparison [0.0]
Evidence Networks can enable Bayesian model comparison when state-of-the-art methods fail.
We introduce the leaky parity-odd power transform, leading to the novel l-POP-Exponential'' loss function.
We show that Evidence Networks are explicitly independent of dimensionality of the parameter space and scale mildly with the complexity of the posterior probability density function.
arXiv Detail & Related papers (2023-05-18T18:14:53Z) - BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen
Neural Networks [50.15201777970128]
We propose BayesCap that learns a Bayesian identity mapping for the frozen model, allowing uncertainty estimation.
BayesCap is a memory-efficient method that can be trained on a small fraction of the original dataset.
We show the efficacy of our method on a wide variety of tasks with a diverse set of architectures.
arXiv Detail & Related papers (2022-07-14T12:50:09Z) - Contributions to Large Scale Bayesian Inference and Adversarial Machine
Learning [0.0]
The rampant adoption of ML methodologies has revealed that models are usually adopted to make decisions without taking into account the uncertainties in their predictions.
We believe that developing ML systems that take into predictive account uncertainties and are robust against adversarial examples is a must for real-world tasks.
arXiv Detail & Related papers (2021-09-25T23:02:47Z) - Evaluating State-of-the-Art Classification Models Against Bayes
Optimality [106.50867011164584]
We show that we can compute the exact Bayes error of generative models learned using normalizing flows.
We use our approach to conduct a thorough investigation of state-of-the-art classification models.
arXiv Detail & Related papers (2021-06-07T06:21:20Z) - Simulation comparisons between Bayesian and de-biased estimators in
low-rank matrix completion [0.0]
We study the low-rank matrix completion problem, a class of machine learning problems, that aims at the prediction of missing entries in a partially observed matrix.
We compare the Bayesian approaches and a recently introduced de-biased estimator which provides a useful way to build confidence intervals of interest.
We find that the empirical coverage rate of the confidence intervals obtained by the de-biased estimator for an entry is absolutely lower than of the considered credible interval.
arXiv Detail & Related papers (2021-03-22T12:02:00Z) - Bayesian Model Averaging for Causality Estimation and its Approximation
based on Gaussian Scale Mixture Distributions [0.0]
We first show from a Bayesian perspective that it is Bayes optimal to weight (average) the causal effects estimated under each model.
We develop an approximation to the Bayes optimal estimator by using Gaussian scale mixture distributions.
arXiv Detail & Related papers (2021-03-15T08:07:58Z) - Nonparametric Estimation of the Fisher Information and Its Applications [82.00720226775964]
This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$.
An estimator proposed by Bhattacharya is revisited and improved convergence rates are derived.
A new estimator, termed a clipped estimator, is proposed.
arXiv Detail & Related papers (2020-05-07T17:21:56Z) - Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
We show that a posterior approximation distinct from the variational distribution should be used for making decisions.
Motivated by these theoretical results, we propose learning several approximate proposals for the best model.
In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
arXiv Detail & Related papers (2020-02-17T19:23:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.