Quantifying Confidence in Assurance 2.0 Arguments

• URL: http://arxiv.org/abs/2604.00034v1
• Date: Sat, 21 Mar 2026 01:54:10 GMT
• Title: Quantifying Confidence in Assurance 2.0 Arguments
• Abstract summary: We present a new method for assessing probabilistic confidence in assurance case arguments that is simple, systematic and sound.<n>It exploits the ways claims are decomposed in a structured argument and provides different approaches according to the different degrees of (in)dependence and diversity among subclaims.<n>We show that the method is not susceptible to the counterexamples that Graydon and Holloway exhibit for other approaches to confidence and we recommend it as an additional tool in evaluation of Assurance 2.0 arguments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Confidence is central to safety and assurance cases: how much confidence a decision requires and how much the argument actually provides are both important questions. We present a new method for assessing probabilistic confidence in assurance case arguments that is simple, systematic and sound. It exploits the ways claims are decomposed in a structured argument and provides different approaches according to the different degrees of (in)dependence and diversity among subclaims and the way they eliminate concerns that undermine confidence in their parent claims. The method uses only elementary probabilistic constructions that are well-known in other contexts (e.g., Frechet bounds) but we interpret and apply them in a manner that is specifically focused on assurance arguments and requires no background in probabilistic analysis. We show that the method is not susceptible to the counterexamples that Graydon and Holloway exhibit for other approaches to confidence and we recommend it as an additional tool in evaluation of Assurance 2.0 arguments. The primary evaluation criteria for Assurance 2.0 remain logical indefeasibility and dialectical examination, but probabilistic assessment can be useful in evaluating cost/confidence tradeoffs for different risk levels, and the overall balance of confidence across a structured argument.

Related papers

• Towards a compositional semantics for quantitative confidence assessment in assurance arguments [0.10499611180329803]
  Existing approaches address structure and soundness but largely reason over truth values, not over confidence in the justification of claims.<n>We propose a confidence semantics that represents argument elements as SL opinions and maps relations between elements to SL operators modelling how confidence flows.
  arXiv  Detail & Related papers  (2026-05-21T09:17:27Z)
• Unified Multimodal Uncertain Inference [81.64340342055327]
  Unified Multimodal Uncertain Inference (UMUI) is a task spanning text, audio, and video.<n>We introduce CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based calibrated confidence probing to produce predictions.
  arXiv  Detail & Related papers  (2026-04-09T18:46:34Z)
• Assessing confidence in frontier AI safety cases [37.839615078345886]
  A safety case presents a structured argument in support of a top-level claim about a safety property of the system.<n>This raises the question of what level of confidence should be associated with a top-level claim.<n>We propose a method by which AI developers can prioritise, and thereby make their investigation of argument defeaters more efficient.
  arXiv  Detail & Related papers  (2025-02-09T06:35:11Z)
• Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences [62.52739672949452]
  Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary.<n>We propose relative confidence estimation, where we match up questions against each other and ask the model to make relative judgments of confidence.<n>Treating each question as a "player" in a series of matchups against other questions and the model's preferences as match outcomes, we can use rank aggregation methods like Elo rating and Bradley-Terry to translate the model's confidence preferences into confidence scores.
  arXiv  Detail & Related papers  (2025-02-03T07:43:27Z)
• Probabilistic Modeling of Disparity Uncertainty for Robust and Efficient Stereo Matching [61.73532883992135]
  We propose a new uncertainty-aware stereo matching framework.<n>We adopt Bayes risk as the measurement of uncertainty and use it to separately estimate data and model uncertainty.
  arXiv  Detail & Related papers  (2024-12-24T23:28:20Z)
• Confidence in Assurance 2.0 Cases [0.0]
  We consider how confidence can be assessed in the rigorous approach we call Assurance 2.0.<n>Our goal is indefeasible confidence and we approach it from four different perspectives.
  arXiv  Detail & Related papers  (2024-09-16T19:00:21Z)
• Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection [90.71323430635593]
  We propose a novel self-detection paradigm that considers the comprehensive answer space beyond LLM-generated answers. Building upon this paradigm, we introduce a two-step framework, which firstly instructs LLM to reflect and provide justifications for each candidate answer. This framework can be seamlessly integrated with existing approaches for superior self-detection.
  arXiv  Detail & Related papers  (2024-03-15T02:38:26Z)
• Binary Classification with Confidence Difference [100.08818204756093]
  This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
  arXiv  Detail & Related papers  (2023-10-09T11:44:50Z)
• Assessing Confidence with Assurance 2.0 [0.0]
  We argue that confidence cannot be reduced to a single attribute or measurement. Positive Perspectives consider the extent to which the evidence and overall argument of the case combine to make a positive statement. Negative Perspectives record doubts and challenges to the case, typically expressed as defeaters. Residual Doubts: the world is uncertain so not all potential defeaters can be resolved.
  arXiv  Detail & Related papers  (2022-05-03T22:10:59Z)
• Handling Epistemic and Aleatory Uncertainties in Probabilistic Circuits [18.740781076082044]
  We propose an approach to overcome the independence assumption behind most of the approaches dealing with a large class of probabilistic reasoning. We provide an algorithm for Bayesian learning from sparse, albeit complete, observations. Each leaf of such circuits is labelled with a beta-distributed random variable that provides us with an elegant framework for representing uncertain probabilities.
  arXiv  Detail & Related papers  (2021-02-22T10:03:15Z)
• An evaluation of word-level confidence estimation for end-to-end automatic speech recognition [70.61280174637913]
  We investigate confidence estimation for end-to-end automatic speech recognition (ASR) We provide an extensive benchmark of popular confidence methods on four well-known speech datasets. Our results suggest a strong baseline can be obtained by scaling the logits by a learnt temperature.
  arXiv  Detail & Related papers  (2021-01-14T09:51:59Z)
• Towards Safe Policy Improvement for Non-Stationary MDPs [48.9966576179679]
  Many real-world problems of interest exhibit non-stationarity, and when stakes are high, the cost associated with a false stationarity assumption may be unacceptable. We take the first steps towards ensuring safety, with high confidence, for smoothly-varying non-stationary decision problems. Our proposed method extends a type of safe algorithm, called a Seldonian algorithm, through a synthesis of model-free reinforcement learning with time-series analysis.
  arXiv  Detail & Related papers  (2020-10-23T20:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.