A Novel Bayes' Theorem for Upper Probabilities

• URL: http://arxiv.org/abs/2307.06831v1
• Date: Thu, 13 Jul 2023 15:50:49 GMT
• Title: A Novel Bayes' Theorem for Upper Probabilities
• Authors: Michele Caprio, Yusuf Sale, Eyke H\"ullermeier, Insup Lee
• Abstract summary: In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes' posterior probability of a measurable set $A$. In this paper, we introduce a generalization of their result by additionally addressing uncertainty related to the likelihood.
• Score: 7.527234046228324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes' posterior probability of a measurable set $A$, when the prior lies in a class of probability measures $\mathcal{P}$ and the likelihood is precise. They also give a sufficient condition for such upper bound to hold with equality. In this paper, we introduce a generalization of their result by additionally addressing uncertainty related to the likelihood. We give an upper bound for the posterior probability when both the prior and the likelihood belong to a set of probabilities. Furthermore, we give a sufficient condition for this upper bound to become an equality. This result is interesting on its own, and has the potential of being applied to various fields of engineering (e.g. model predictive control), machine learning, and artificial intelligence.

Related papers

• Online Prediction of Stochastic Sequences with High Probability Regret Bounds [16.68585810113338]
  We revisit the classical problem of universal prediction of sequences with a finite time horizon $T$ known to the learner.<n>We propose vanishing regret bounds that hold with high probability, complementing existing bounds from the literature that hold in expectation.<n>For the case of universal prediction of a process over a countable alphabet, our bound states a convergence rate of $mathcalO(T-1/2 -1/2)$ with probability as least $1-$ compared to prior known in-expectation bounds of the order $mathcalO(T-1/2)$.
  arXiv  Detail & Related papers  (2026-02-18T07:26:37Z)
• A Refinement of Vapnik--Chervonenkis' Theorem [0.0]
  Vapnik--Chervonenkis' theorem is a seminal result in machine learning.<n>We revisit the probabilistic component of the classical argument.
  arXiv  Detail & Related papers  (2026-01-23T02:57:29Z)
• Certain but not Probable? Differentiating Certainty from Probability in LLM Token Outputs for Probabilistic Scenarios [1.1510009152620668]
  We investigate the relationship between token certainty and alignment with theoretical probability distributions in well-defined probabilistic scenarios.<n>We measure two dimensions: (1) response validity with respect to scenario constraints, and (2) alignment between token-level output probabilities and theoretical probabilities.<n>Our results indicate that, while both models achieve perfect in-domain response accuracy across all prompt scenarios, their token-level probability and entropy values consistently diverge from the corresponding theoretical distributions.
  arXiv  Detail & Related papers  (2025-11-01T16:51:11Z)
• Optimal Conformal Prediction under Epistemic Uncertainty [61.46247583794497]
  Conformal prediction (CP) is a popular framework for representing uncertainty.<n>We introduce Bernoulli prediction sets (BPS) which produce the smallest prediction sets that ensure conditional coverage.<n>When given first-order predictions, BPS reduces to the well-known adaptive prediction sets (APS)
  arXiv  Detail & Related papers  (2025-05-25T08:32:44Z)
• Can a Bayesian Oracle Prevent Harm from an Agent? [48.12936383352277]
  We consider estimating a context-dependent bound on the probability of violating a given safety specification. Noting that different plausible hypotheses about the world could produce very different outcomes, we derive on the safety violation probability predicted under the true but unknown hypothesis. We consider two forms of this result, in the iid case and in the non-iid case, and conclude with open problems towards turning such results into practical AI guardrails.
  arXiv  Detail & Related papers  (2024-08-09T18:10:42Z)
• On The Statistical Representation Properties Of The Perturb-Softmax And The Perturb-Argmax Probability Distributions [17.720298535412443]
  Gumbel-Softmax and Gumbel-Argmax probability distributions are useful in learning discrete structures in discriminative learning. Despite the efforts invested in optimizing these probability models, their statistical properties are under-explored. We investigate their representation properties and determine for which families of parameters these probability distributions are complete. We conclude the analysis by identifying two sets of parameters that satisfy these assumptions and thus admit a complete and minimal representation.
  arXiv  Detail & Related papers  (2024-06-04T10:22:12Z)
• A new approach for imprecise probabilities [0.0]
  We characterize a broad class of interval probability measures and define their properties. As a byproduct, a formal solution to the century-old Keynes-Ramsey controversy is presented.
  arXiv  Detail & Related papers  (2024-02-04T16:09:04Z)
• A Priori Determination of the Pretest Probability [0.0]
  We introduce a novel method to estimate the pretest probability of disease, a priori, utilizing the Logit function from the logistic regression model. In a patient presenting with signs or symptoms, the minimal bound of the pretest probability, $phi$, can be approximated by: $phi approx frac15lnleft[styleprod_theta=1ikappa_thetaright]$ where $ln$ is the natural, and $kappa_theta$ is the likelihood ratio associated with
  arXiv  Detail & Related papers  (2024-01-08T18:44:43Z)
• Predicting generalization performance with correctness discriminators [64.00420578048855]
  We present a novel model that establishes upper and lower bounds on the accuracy, without requiring gold labels for the unseen data. We show across a variety of tagging, parsing, and semantic parsing tasks that the gold accuracy is reliably between the predicted upper and lower bounds.
  arXiv  Detail & Related papers  (2023-11-15T22:43:42Z)
• Estimating Optimal Policy Value in General Linear Contextual Bandits [50.008542459050155]
  In many bandit problems, the maximal reward achievable by a policy is often unknown in advance. We consider the problem of estimating the optimal policy value in the sublinear data regime before the optimal policy is even learnable. We present a more practical, computationally efficient algorithm that estimates a problem-dependent upper bound on $V*$.
  arXiv  Detail & Related papers  (2023-02-19T01:09:24Z)
• Relative Probability on Finite Outcome Spaces: A Systematic Examination of its Axiomatization, Properties, and Applications [0.0]
  This work proposes a view of probability as a relative measure rather than an absolute one. We focus on finite outcome spaces and develop three fundamental axioms that establish requirements for relative probability functions.
  arXiv  Detail & Related papers  (2022-12-30T05:16:57Z)
• Reconciling Individual Probability Forecasts [78.0074061846588]
  We show that two parties who agree on the data cannot disagree on how to model individual probabilities. We conclude that although individual probabilities are unknowable, they are contestable via a computationally and data efficient process.
  arXiv  Detail & Related papers  (2022-09-04T20:20:35Z)
• High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize [55.0090961425708]
  We propose a new, simplified high probability analysis of AdaGrad for smooth, non- probability problems. We present our analysis in a modular way and obtain a complementary $mathcal O (1 / TT)$ convergence rate in the deterministic setting. To the best of our knowledge, this is the first high probability for AdaGrad with a truly adaptive scheme, i.e., completely oblivious to the knowledge of smoothness.
  arXiv  Detail & Related papers  (2022-04-06T13:50:33Z)
• Attainability and Optimality: The Equalized Odds Fairness Revisited [8.44348159032116]
  We consider the attainability of the Equalized Odds notion of fairness. For classification, we prove that compared to enforcing fairness by post-processing, one can always benefit from exploiting all available features. While performance prediction can attain Equalized Odds with theoretical guarantees, we also discuss its limitation and potential negative social impacts.
  arXiv  Detail & Related papers  (2022-02-24T01:30:31Z)
• Distributionally Robust Bayesian Quadrature Optimization [60.383252534861136]
  We study BQO under distributional uncertainty in which the underlying probability distribution is unknown except for a limited set of its i.i.d. samples. A standard BQO approach maximizes the Monte Carlo estimate of the true expected objective given the fixed sample set. We propose a novel posterior sampling based algorithm, namely distributionally robust BQO (DRBQO) for this purpose.
  arXiv  Detail & Related papers  (2020-01-19T12:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.