Inferential Moments of Uncertain Multivariable Systems

• URL: http://arxiv.org/abs/2305.01841v2
• Date: Fri, 17 Nov 2023 16:37:44 GMT
• Title: Inferential Moments of Uncertain Multivariable Systems
• Authors: Kevin Vanslette
• Abstract summary: We treat Bayesian probability updating as a random process and uncover intrinsic quantitative features of joint probability distributions called inferential moments. Inferential moments quantify shape information about how a prior distribution is expected to update in response to yet to be obtained information. We find a power series expansion of the mutual information in terms of inferential moments, which implies a connection between inferential theoretic logic and elements of information theory.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random process and uncover intrinsic quantitative features of joint probability distributions called inferential moments. Inferential moments quantify shape information about how a prior distribution is expected to update in response to yet to be obtained information. Further, we quantify the unique probability distribution whose statistical moments are the inferential moments in question. We find a power series expansion of the mutual information in terms of inferential moments, which implies a connection between inferential theoretic logic and elements of information theory. Of particular interest is the inferential deviation, which is the expected variation of the probability of one variable in response to an inferential update of another. We explore two applications that analyze the inferential deviations of a Bayesian network to improve decision-making. We implement simple greedy algorithms for exploring sensor tasking using inferential deviations that generally outperform similar greedy mutual information algorithms in terms of root mean squared error between epistemic probability estimates and the ground truth probabilities they are estimating.

Related papers

• Intervention and Conditioning in Causal Bayesian Networks [23.225006087292765]
  We show that by making simple yet often realistic independence assumptions, it is possible to estimate the probability of an interventional formula. In many cases of interest, when the assumptions are appropriate, these probability estimates can be evaluated using observational data.
  arXiv  Detail & Related papers  (2024-05-23T15:55:38Z)
• Tractable Function-Space Variational Inference in Bayesian Neural Networks [72.97620734290139]
  A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters. We propose a scalable function-space variational inference method that allows incorporating prior information. We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks.
  arXiv  Detail & Related papers  (2023-12-28T18:33:26Z)
• On the Properties and Estimation of Pointwise Mutual Information Profiles [49.877314063833296]
  The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. We introduce a novel family of distributions, Bend and Mix Models, for which the profile can be accurately estimated using Monte Carlo methods.
  arXiv  Detail & Related papers  (2023-10-16T10:02:24Z)
• Beyond Normal: On the Evaluation of Mutual Information Estimators [52.85079110699378]
  We show how to construct a diverse family of distributions with known ground-truth mutual information. We provide guidelines for practitioners on how to select appropriate estimator adapted to the difficulty of problem considered.
  arXiv  Detail & Related papers  (2023-06-19T17:26:34Z)
• Statistical Inference Under Constrained Selection Bias [20.862583584531322]
  We propose a framework that enables statistical inference in the presence of selection bias. The output is high-probability bounds on the value of an estimand for the target distribution. We analyze the computational and statistical properties of methods to estimate these bounds and show that our method can produce informative bounds on a variety of simulated and semisynthetic tasks.
  arXiv  Detail & Related papers  (2023-06-05T23:05:26Z)
• A Bayesian Framework for Information-Theoretic Probing [51.98576673620385]
  We argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. This paper proposes a new framework to measure what we term Bayesian mutual information.
  arXiv  Detail & Related papers  (2021-09-08T18:08:36Z)
• Entropy, Information, and the Updating of Probabilities [0.0]
  This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The ME method goes beyond the mere selection of a single posterior, but also addresses the question of how much less probable other distributions might be.
  arXiv  Detail & Related papers  (2021-07-09T16:27:23Z)
• BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
  We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable. We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-06-07T10:14:18Z)
• Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
  We describe two types of forecasting problems: regular grid-based and graph-based. We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
  arXiv  Detail & Related papers  (2021-05-25T14:35:46Z)
• Handling Epistemic and Aleatory Uncertainties in Probabilistic Circuits [18.740781076082044]
  We propose an approach to overcome the independence assumption behind most of the approaches dealing with a large class of probabilistic reasoning. We provide an algorithm for Bayesian learning from sparse, albeit complete, observations. Each leaf of such circuits is labelled with a beta-distributed random variable that provides us with an elegant framework for representing uncertain probabilities.
  arXiv  Detail & Related papers  (2021-02-22T10:03:15Z)
• DEMI: Discriminative Estimator of Mutual Information [5.248805627195347]
  Estimating mutual information between continuous random variables is often intractable and challenging for high-dimensional data. Recent progress has leveraged neural networks to optimize variational lower bounds on mutual information. Our approach is based on training a classifier that provides the probability that a data sample pair is drawn from the joint distribution.
  arXiv  Detail & Related papers  (2020-10-05T04:19:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.