Assessing Confidence with Assurance 2.0

• URL: http://arxiv.org/abs/2205.04522v4
• Date: Fri, 3 May 2024 05:36:36 GMT
• Title: Assessing Confidence with Assurance 2.0
• Authors: Robin Bloomfield, John Rushby,
• Abstract summary: We argue that confidence cannot be reduced to a single attribute or measurement. Positive Perspectives consider the extent to which the evidence and overall argument of the case combine to make a positive statement. Negative Perspectives record doubts and challenges to the case, typically expressed as defeaters. Residual Doubts: the world is uncertain so not all potential defeaters can be resolved.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: An assurance case is intended to provide justifiable confidence in the truth of its top claim, which typically concerns safety or security. A natural question is then "how much" confidence does the case provide? We argue that confidence cannot be reduced to a single attribute or measurement. Instead, we suggest it should be based on attributes that draw on three different perspectives: positive, negative, and residual doubts. Positive Perspectives consider the extent to which the evidence and overall argument of the case combine to make a positive statement justifying belief in its claims. We set a high bar for justification, requiring it to be indefeasible. The primary positive measure for this is soundness, which interprets the argument as a logical proof. Confidence in evidence can be expressed probabilistically and we use confirmation measures to ensure that the "weight" of evidence crosses some threshold. In addition, probabilities can be aggregated from evidence through the steps of the argument using probability logics to yield what we call probabilistic valuations for the claims. Negative Perspectives record doubts and challenges to the case, typically expressed as defeaters, and their exploration and resolution. Assurance developers must guard against confirmation bias and should vigorously explore potential defeaters as they develop the case, and should record them and their resolution to avoid rework and to aid reviewers. Residual Doubts: the world is uncertain so not all potential defeaters can be resolved. We explore risks and may deem them acceptable or unavoidable. It is crucial however that these judgments are conscious ones and that they are recorded in the assurance case. This report examines the perspectives in detail and indicates how Clarissa, our prototype toolset for Assurance 2.0, assists in their evaluation.

Related papers

• Uncertainty Quantification in Stereo Matching [61.73532883992135]
  We propose a new framework for stereo matching and its uncertainty quantification. We adopt Bayes risk as a measure of uncertainty and estimate data and model uncertainty separately. We apply our uncertainty method to improve prediction accuracy by selecting data points with small uncertainties.
  arXiv  Detail & Related papers  (2024-12-24T23:28:20Z)
• Correctness is not Faithfulness in RAG Attributions [47.48625339105129]
  Explicitly citing source documents allows users to verify generated responses and increases trust. Prior work largely evaluates citation correctness - whether cited documents support the corresponding statements. To establish trust in attributed answers, we must examine both citation correctness and citation faithfulness.
  arXiv  Detail & Related papers  (2024-12-23T21:57:11Z)
• Confidence in the Reasoning of Large Language Models [0.0]
  Confidence is measured in terms of persistence in keeping their answer when prompted to reconsider. Confidence is only partially explained by the underlying token-level probability.
  arXiv  Detail & Related papers  (2024-12-19T10:04:29Z)
• On the Robustness of Adversarial Training Against Uncertainty Attacks [9.180552487186485]
  In learning problems, the noise inherent to the task at hand hinders the possibility to infer without a certain degree of uncertainty. In this work, we reveal both empirically and theoretically that defending against adversarial examples, i.e., carefully perturbed samples that cause misclassification, guarantees a more secure, trustworthy uncertainty estimate. To support our claims, we evaluate multiple adversarial-robust models from the publicly available benchmark RobustBench on the CIFAR-10 and ImageNet datasets.
  arXiv  Detail & Related papers  (2024-10-29T11:12:44Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users. We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions. We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Confidence in Assurance 2.0 Cases [0.0]
  We consider how confidence can be assessed in the rigorous approach we call Assurance 2.0. Our goal is indefeasible confidence and we approach it from four different perspectives.
  arXiv  Detail & Related papers  (2024-09-16T19:00:21Z)
• Defeaters and Eliminative Argumentation in Assurance 2.0 [0.0]
  This report describes how defeaters, and multiple levels of defeaters, should be represented and assessed in Assurance 2.0. A valid concern about this process is that human judgement is fallible and prone to confirmation bias.
  arXiv  Detail & Related papers  (2024-05-16T22:10:01Z)
• Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations [63.330182403615886]
  A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability. Three situations where this is particularly apparent are correctness, hallucinations when given unanswerable questions, and safety. In all three cases, models should ideally abstain from responding, much like humans, whose ability to understand uncertainty makes us refrain from answering questions we don't know.
  arXiv  Detail & Related papers  (2024-04-16T23:56:38Z)
• Did You Mean...? Confidence-based Trade-offs in Semantic Parsing [52.28988386710333]
  We show how a calibrated model can help balance common trade-offs in task-oriented parsing. We then examine how confidence scores can help optimize the trade-off between usability and safety.
  arXiv  Detail & Related papers  (2023-03-29T17:07:26Z)
• Increasing Confidence in Adversarial Robustness Evaluations [53.2174171468716]
  We propose a test to identify weak attacks and thus weak defense evaluations. Our test slightly modifies a neural network to guarantee the existence of an adversarial example for every sample. For eleven out of thirteen previously-published defenses, the original evaluation of the defense fails our test, while stronger attacks that break these defenses pass it.
  arXiv  Detail & Related papers  (2022-06-28T13:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.