Mathematical Theory of Bayesian Statistics for Unknown Information Source

• URL: http://arxiv.org/abs/2206.05630v1
• Date: Sat, 11 Jun 2022 23:35:06 GMT
• Title: Mathematical Theory of Bayesian Statistics for Unknown Information Source
• Authors: Sumio Watanabe
• Abstract summary: In statistical inference, uncertainty is unknown and all models are wrong. We show general properties of cross validation, information criteria, and marginal likelihood. The derived theory holds even if an unknown uncertainty is unrealizable by a statistical morel or even if the posterior distribution cannot be approximated by any normal distribution.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In statistical inference, uncertainty is unknown and all models are wrong. A person who makes a statistical model and a prior distribution is simultaneously aware that they are fictional and virtual candidates. In order to study such cases, several statistical measures have been constructed, such as cross validation, information criteria, and marginal likelihood, however, their mathematical properties have not yet been completely clarified when statistical models are under- and over- parametrized. In this paper, we introduce a place of mathematical theory of Bayesian statistics for unknown uncertainty, on which we show general properties of cross validation, information criteria, and marginal likelihood. The derived theory holds even if an unknown uncertainty is unrealizable by a statistical morel or even if the posterior distribution cannot be approximated by any normal distribution, hence it gives a helpful standpoint for a person who cannot believe in any specific model and prior. The results are followings. (1) There exists a more precise statistical measure of the generalization loss than leave-one-out cross validation and information criterion based on the mathematical properties of them. (2) There exists a more efficient approximation method of the free energy, which is the minus log marginal likelihood, even if the posterior distribution cannot be approximated by any normal distribution. (3) And the prior distributions optimized by the cross validation and the widely applicable information criterion are asymptotically equivalent to each other, which are different from that by the marginal likelihood.

Related papers

• Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances [2.317910166616341]
  In statistical inference, we commonly assume that samples are independent and identically distributed from a probability distribution. In this paper, we explore the application of GAN-like estimators to a general class of statistical models.
  arXiv  Detail & Related papers  (2024-10-28T05:50:47Z)
• Transformer-based Parameter Estimation in Statistics [0.0]
  We propose a transformer-based approach to parameter estimation. It does not even require knowing the probability density function, which is needed by numerical methods. It is shown that our approach achieves similar or better accuracy as measured by mean-square-errors.
  arXiv  Detail & Related papers  (2024-02-28T04:30:41Z)
• User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems [49.75149094527068]
  We show that diffusion models can be adapted to make predictions and provide uncertainty quantification for chaotic dynamical systems. We develop a probabilistic approximation scheme for the conditional score function which converges to the true distribution as the noise level decreases. We are able to sample conditionally on nonlinear userdefined events at inference time, and matches data statistics even when sampling from the tails of the distribution.
  arXiv  Detail & Related papers  (2023-06-13T03:42:03Z)
• Sufficient Identification Conditions and Semiparametric Estimation under Missing Not at Random Mechanisms [4.211128681972148]
  Conducting valid statistical analyses is challenging in the presence of missing-not-at-random (MNAR) data. We consider a MNAR model that generalizes several prior popular MNAR models in two ways. We propose methods for testing the independence restrictions encoded in such models using odds ratio as our parameter of interest.
  arXiv  Detail & Related papers  (2023-06-10T13:46:16Z)
• Recent Advances in Algebraic Geometry and Bayesian Statistics [0.0]
  This article is a review of theoretical advances in the research field of algebraic geometry and Bayesian statistics. Two mathematical solutions and three applications to statistics based on algebraic geometry reported in this article are now being used in many practical fields in data science and artificial intelligence.
  arXiv  Detail & Related papers  (2022-11-18T06:19:05Z)
• Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
  We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
  arXiv  Detail & Related papers  (2022-10-03T06:09:01Z)
• Statistical Properties of the Entropy from Ordinal Patterns [55.551675080361335]
  Knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class of time series models would allow statistical tests that are unavailable to date. We characterize the distribution of the empirical Shannon's Entropy for any model under which the true normalized Entropy is neither zero nor one. We present a bilateral test that verifies if there is enough evidence to reject the hypothesis that two signals produce ordinal patterns with the same Shannon's Entropy.
  arXiv  Detail & Related papers  (2022-09-15T23:55:58Z)
• Uncertainty Modeling for Out-of-Distribution Generalization [56.957731893992495]
  We argue that the feature statistics can be properly manipulated to improve the generalization ability of deep learning models. Common methods often consider the feature statistics as deterministic values measured from the learned features. We improve the network generalization ability by modeling the uncertainty of domain shifts with synthesized feature statistics during training.
  arXiv  Detail & Related papers  (2022-02-08T16:09:12Z)
• Characterizations of non-normalized discrete probability distributions and their application in statistics [0.0]
  We derive explicit formulae for the mass functions of discrete probability laws that identify those distributions. Our characterizations, and hence the applications built on them, do not require any knowledge about normalization constants of the probability laws.
  arXiv  Detail & Related papers  (2020-11-09T12:08:12Z)
• Contextuality scenarios arising from networks of stochastic processes [68.8204255655161]
  An empirical model is said contextual if its distributions cannot be obtained marginalizing a joint distribution over X. We present a different and classical source of contextual empirical models: the interaction among many processes. The statistical behavior of the network in the long run makes the empirical model generically contextual and even strongly contextual.
  arXiv  Detail & Related papers  (2020-06-22T16:57:52Z)
• Density of States Estimation for Out-of-Distribution Detection [69.90130863160384]
  DoSE is the density of states estimator. We demonstrate DoSE's state-of-the-art performance against other unsupervised OOD detectors.
  arXiv  Detail & Related papers  (2020-06-16T16:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.