Enhancing Covid-19 Decision-Making by Creating an Assurance Case for Simulation Models

• URL: http://arxiv.org/abs/2005.08381v1
• Date: Sun, 17 May 2020 22:07:05 GMT
• Title: Enhancing Covid-19 Decision-Making by Creating an Assurance Case for Simulation Models
• Authors: Ibrahim Habli, Rob Alexander, Richard Hawkins, Mark Sujan, John McDermid, Chiara Picardi, Tom Lawton
• Abstract summary: We argue that any COVID-19 simulation model that is used to guide critical policy decisions would benefit from being supported with an assurance case. This would enable a critical review of the implicit assumptions and inherent uncertainty in modelling, and would give the overall decision-making process greater transparency and accountability.
• Score: 7.241250079741012
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Simulation models have been informing the COVID-19 policy-making process. These models, therefore, have significant influence on risk of societal harms. But how clearly are the underlying modelling assumptions and limitations communicated so that decision-makers can readily understand them? When making claims about risk in safety-critical systems, it is common practice to produce an assurance case, which is a structured argument supported by evidence with the aim to assess how confident we should be in our risk-based decisions. We argue that any COVID-19 simulation model that is used to guide critical policy decisions would benefit from being supported with such a case to explain how, and to what extent, the evidence from the simulation can be relied on to substantiate policy conclusions. This would enable a critical review of the implicit assumptions and inherent uncertainty in modelling, and would give the overall decision-making process greater transparency and accountability.

Related papers

• Case-based Reasoning Augmented Large Language Model Framework for Decision Making in Realistic Safety-Critical Driving Scenarios [1.5367554212163714]
  This paper presents a Case-Based Reasoning Augmented Large Language Model (CBR-LLM) framework for evasive maneuver decision-making in complex risk scenarios.<n>Our approach integrates semantic scene understanding from dashcam video inputs with the retrieval of relevant past driving cases.<n>Experiments show that our framework improves decision accuracy, justification quality, and alignment with human expert behavior.
  arXiv  Detail & Related papers  (2025-06-25T15:19:25Z)
• Marginal Fairness: Fair Decision-Making under Risk Measures [24.99817090886293]
  This paper introduces marginal fairness, a new individual fairness notion for equitable decision-making in the presence of protected attributes.<n>We model business decision-making in highly regulated industries (such as insurance and finance) as a two-step process.<n>A numerical study and an empirical implementation using an auto insurance dataset demonstrate how the framework can be applied in practice.
  arXiv  Detail & Related papers  (2025-05-24T22:44:35Z)
• Answer, Refuse, or Guess? Investigating Risk-Aware Decision Making in Language Models [63.54557575233165]
  We formalize the task of risk-aware decision-making, expose critical weaknesses in existing LMs, and propose skill-decomposition solutions. Our findings show that even cutting-edge LMs--both regular and reasoning models--still require explicit prompt chaining to handle the task effectively.
  arXiv  Detail & Related papers  (2025-03-03T09:16:26Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. Our research identifies two critical latent factors affecting RAG's confidence in its predictions. We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference [3.422309388045878]
  Large language models (LLMs) such as ChatGPT are known to pose important risks. misplaced confidence arises from over-confidence or under-confidence, that the models have in their inference. We propose an experimental framework consisting of a two-level inference architecture and appropriate metrics for measuring such risks.
  arXiv  Detail & Related papers  (2024-08-04T05:24:32Z)
• Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction [55.77015419028725]
  We develop methods that permit valid control of risk when threshold and tradeoff parameters are chosen adaptively. Our methodology supports monotone and nearly-monotone risks, but otherwise makes no distributional assumptions.
  arXiv  Detail & Related papers  (2024-03-28T17:28:06Z)
• On the Societal Impact of Open Foundation Models [93.67389739906561]
  We focus on open foundation models, defined here as those with broadly available model weights. We identify five distinctive properties of open foundation models that lead to both their benefits and risks.
  arXiv  Detail & Related papers  (2024-02-27T16:49:53Z)
• Explaining by Imitating: Understanding Decisions by Interpretable Policy Learning [72.80902932543474]
  Understanding human behavior from observed data is critical for transparency and accountability in decision-making. Consider real-world settings such as healthcare, in which modeling a decision-maker's policy is challenging. We propose a data-driven representation of decision-making behavior that inheres transparency by design, accommodates partial observability, and operates completely offline.
  arXiv  Detail & Related papers  (2023-10-28T13:06:14Z)
• Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learning [76.98542249776257]
  Large-scale language models often face the challenge of "hallucination" We introduce an uncertainty-aware in-context learning framework to empower the model to enhance or reject its output in response to uncertainty.
  arXiv  Detail & Related papers  (2023-10-07T12:06:53Z)
• Analyzing Intentional Behavior in Autonomous Agents under Uncertainty [3.0099979365586265]
  Principled accountability for autonomous decision-making in uncertain environments requires distinguishing intentional outcomes from negligent designs from actual accidents. We propose analyzing the behavior of autonomous agents through a quantitative measure of the evidence of intentional behavior. In a case study, we show how our method can distinguish between 'intentional' and 'accidental' traffic collisions.
  arXiv  Detail & Related papers  (2023-07-04T07:36:11Z)
• Dual policy as self-model for planning [71.73710074424511]
  We refer to the model used to simulate one's decisions as the agent's self-model. Inspired by current reinforcement learning approaches and neuroscience, we explore the benefits and limitations of using a distilled policy network as the self-model.
  arXiv  Detail & Related papers  (2023-06-07T13:58:45Z)
• A Meta-heuristic Approach to Estimate and Explain Classifier Uncertainty [0.4264192013842096]
  This work proposes a set of class-independent meta-heuristics that can characterize the complexity of an instance in terms of factors are mutually relevant to both human and machine learning decision-making. The proposed measures and framework hold promise for improving model development for more complex instances, as well as providing a new means of model abstention and explanation.
  arXiv  Detail & Related papers  (2023-04-20T13:09:28Z)
• Predictive Capability Maturity Quantification using Bayesian Network [0.0]
  In nuclear engineering, modeling and simulations (M&Ss) are widely applied to support risk-informed safety analysis. Due to data gaps, validation becomes a decision-making process under uncertainties. This paper suggests a framework "Predictive Capability Maturity Quantification using Bayesian network (PCMQBN)" as a quantified framework for assessing simulation adequacy.
  arXiv  Detail & Related papers  (2020-08-31T17:09:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.