Approximate Bayesian Computation via Classification
- URL: http://arxiv.org/abs/2111.11507v1
- Date: Mon, 22 Nov 2021 20:07:55 GMT
- Title: Approximate Bayesian Computation via Classification
- Authors: Yuexi Wang, Tetsuya Kaji and Veronika Ro\v{c}kov\'a
- Abstract summary: Approximate Computation (ABC) enables statistical inference in complex models whose likelihoods are difficult to calculate but easy to simulate from.
ABC constructs a kernel-type approximation to the posterior distribution through an accept/reject mechanism which compares summary statistics of real and simulated data.
We consider the traditional accept/reject kernel as well as an exponential weighting scheme which does not require the ABC acceptance threshold.
- Score: 0.966840768820136
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Approximate Bayesian Computation (ABC) enables statistical inference in
complex models whose likelihoods are difficult to calculate but easy to
simulate from. ABC constructs a kernel-type approximation to the posterior
distribution through an accept/reject mechanism which compares summary
statistics of real and simulated data. To obviate the need for summary
statistics, we directly compare empirical distributions with a Kullback-Leibler
(KL) divergence estimator obtained via classification. In particular, we blend
flexible machine learning classifiers within ABC to automate fake/real data
comparisons. We consider the traditional accept/reject kernel as well as an
exponential weighting scheme which does not require the ABC acceptance
threshold. Our theoretical results show that the rate at which our ABC
posterior distributions concentrate around the true parameter depends on the
estimation error of the classifier. We derive limiting posterior shape results
and find that, with a properly scaled exponential kernel, asymptotic normality
holds. We demonstrate the usefulness of our approach on simulated examples as
well as real data in the context of stock volatility estimation.
Related papers
- Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances [2.317910166616341]
In statistical inference, we commonly assume that samples are independent and identically distributed from a probability distribution.
In this paper, we explore the application of GAN-like estimators to a general class of statistical models.
arXiv Detail & Related papers (2024-10-28T05:50:47Z) - Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data [9.913418444556486]
We show how iterative methods can be used to reduce the computational costs for calculating likelihoods, gradients, and predictive distributions with FSAs.
We also present a novel, accurate, and fast way to calculate predictive variances relying on estimations and iterative methods.
All methods are implemented in a free C++ software library with high-level Python and R packages.
arXiv Detail & Related papers (2024-05-23T12:25:22Z) - Statistical Inference with Stochastic Gradient Methods under
$\phi$-mixing Data [9.77185962310918]
We propose a mini-batch SGD estimator for statistical inference when the data is $phi$-mixing.
The confidence intervals are constructed using an associated mini-batch SGD procedure.
The proposed method is memory-efficient and easy to implement in practice.
arXiv Detail & Related papers (2023-02-24T16:16:43Z) - Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain.
We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions.
We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
arXiv Detail & Related papers (2022-11-30T05:33:29Z) - Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated.
We formalize these results both in the sample regime and in the finite regime.
arXiv Detail & Related papers (2022-10-03T06:09:01Z) - Learning Summary Statistics for Bayesian Inference with Autoencoders [58.720142291102135]
We use the inner dimension of deep neural network based Autoencoders as summary statistics.
To create an incentive for the encoder to encode all the parameter-related information but not the noise, we give the decoder access to explicit or implicit information that has been used to generate the training data.
arXiv Detail & Related papers (2022-01-28T12:00:31Z) - Evaluating State-of-the-Art Classification Models Against Bayes
Optimality [106.50867011164584]
We show that we can compute the exact Bayes error of generative models learned using normalizing flows.
We use our approach to conduct a thorough investigation of state-of-the-art classification models.
arXiv Detail & Related papers (2021-06-07T06:21:20Z) - Minimax Quasi-Bayesian estimation in sparse canonical correlation
analysis via a Rayleigh quotient function [1.0878040851638]
Existing rate-optimal estimators for sparse canonical vectors have high computational cost.
We propose a quasi-Bayesian estimation procedure that achieves the minimax estimation rate.
We use the proposed methodology to maximally correlate clinical variables and proteomic data for better understanding the Covid-19 disease.
arXiv Detail & Related papers (2020-10-16T21:00:57Z) - Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers.
We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model.
Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
arXiv Detail & Related papers (2020-06-22T21:12:31Z) - $\gamma$-ABC: Outlier-Robust Approximate Bayesian Computation Based on a
Robust Divergence Estimator [95.71091446753414]
We propose to use a nearest-neighbor-based $gamma$-divergence estimator as a data discrepancy measure.
Our method achieves significantly higher robustness than existing discrepancy measures.
arXiv Detail & Related papers (2020-06-13T06:09:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.