A Methodology for Quantitative AI Risk Modeling

• URL: http://arxiv.org/abs/2512.08844v2
• Date: Thu, 11 Dec 2025 10:25:12 GMT
• Title: A Methodology for Quantitative AI Risk Modeling
• Authors: Malcolm Murray, Steve Barrett, Henry Papadatos, Otter Quarks, Matt Smith, Alejandro Tlaie Boria, Chloé Touzet, Siméon Campos,
• Abstract summary: This paper advances the risk modeling component of AI risk management by introducing a methodology that integrates scenario building with quantitative risk estimation.<n>Our methodology is designed to be applicable to key systemic AI risks, including cyber offense, biological weapon development, harmful manipulation, and loss-of-control.
• Score: 32.594929429306774
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although general-purpose AI systems offer transformational opportunities in science and industry, they simultaneously raise critical concerns about safety, misuse, and potential loss of control. Despite these risks, methods for assessing and managing them remain underdeveloped. Effective risk management requires systematic modeling to characterize potential harms, as emphasized in frameworks such as the EU General-Purpose AI Code of Practice. This paper advances the risk modeling component of AI risk management by introducing a methodology that integrates scenario building with quantitative risk estimation, drawing on established approaches from other high-risk industries. Our methodology models risks through a six-step process: (1) defining risk scenarios, (2) decomposing them into quantifiable parameters, (3) quantifying baseline risk without AI models, (4) identifying key risk indicators such as benchmarks, (5) mapping these indicators to model parameters to estimate LLM uplift, and (6) aggregating individual parameters into risk estimates that enable concrete claims (e.g., X% probability of >\$Y in annual cyber damages). We examine the choices that underlie our methodology throughout the article, with discussions of strengths, limitations, and implications for future research. Our methodology is designed to be applicable to key systemic AI risks, including cyber offense, biological weapon development, harmful manipulation, and loss-of-control, and is validated through extensive application in LLM-enabled cyber offense. Detailed empirical results and cyber-specific insights are presented in a companion paper.

Related papers

• Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5 [61.787178868669265]
  This technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication.<n>This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
  arXiv  Detail & Related papers  (2026-02-16T04:30:06Z)
• Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse [50.87630846876635]
  We develop nine detailed cyber risk models.<n>Each model decomposes attacks into steps using the MITRE ATT&CK framework.<n>Individual estimates are aggregated through Monte Carlo simulation.
  arXiv  Detail & Related papers  (2025-12-09T17:54:17Z)
• The Role of Risk Modeling in Advanced AI Risk Management [33.357295564462284]
  Rapidly advancing artificial intelligence (AI) systems introduce novel, uncertain, and potentially catastrophic risks.<n>Managing these risks requires a mature risk-management infrastructure whose cornerstone is rigorous risk modeling.<n>We argue that advanced-AI governance should adopt a similar dual approach and that verifiable, provably-safe AI architectures are urgently needed.
  arXiv  Detail & Related papers  (2025-12-09T15:37:33Z)
• RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration [81.38705556267917]
  Existing safety evaluation methods for large language models (LLMs) suffer from inherent limitations.<n>We introduce a theoretical framework that reconstructs the underlying risk concept space.<n>We propose RADAR, a multi-agent collaborative evaluation framework.
  arXiv  Detail & Related papers  (2025-09-28T09:35:32Z)
• Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks [0.0]
  This paper examines six commonly discussed AI catastrophic risks: CBRN, cyber offense, sudden loss of control, gradual loss of control, environmental risk, and geopolitical risk.<n>We characterize these risks across seven key dimensions, namely intent, competency, entity, polarity, linearity, reach, and order.<n>We conduct risk pathway modeling by mapping step-by-step progressions from the initial hazard to the resulting harms.
  arXiv  Detail & Related papers  (2025-08-08T15:56:05Z)
• From nuclear safety to LLM security: Applying non-probabilistic risk management strategies to build safe and secure LLM-powered systems [49.1574468325115]
  Large language models (LLMs) offer unprecedented and growing capabilities, but also introduce complex safety and security challenges.<n>Previous research found that risk management in various fields of engineering such as nuclear or civil engineering is often solved by generic (i.e. field-agnostic) strategies.<n>Here we show how emerging risks in LLM-powered systems could be met with 100+ of these non-probabilistic strategies to risk management.
  arXiv  Detail & Related papers  (2025-05-20T16:07:41Z)
• Adapting Probabilistic Risk Assessment for AI [0.0]
  General-purpose artificial intelligence (AI) systems present an urgent risk management challenge.<n>Current methods often rely on selective testing and undocumented assumptions about risk priorities.<n>This paper introduces the probabilistic risk assessment (PRA) for AI framework.
  arXiv  Detail & Related papers  (2025-04-25T17:59:14Z)
• Mapping AI Benchmark Data to Quantitative Risk Estimates Through Expert Elicitation [0.7889270818022226]
  We show how existing AI benchmarks can be used to facilitate the creation of risk estimates.<n>We describe the results of a pilot study in which experts use information from Cybench, an AI benchmark, to generate probability estimates.
  arXiv  Detail & Related papers  (2025-03-06T10:39:47Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorization [49.26510528455664]
  We introduce the Risk-sensitive Individual-Global-Max (RIGM) principle as a generalization of the Individual-Global-Max (IGM) and Distributional IGM (DIGM) principles. We show that RiskQ can obtain promising performance through extensive experiments.
  arXiv  Detail & Related papers  (2023-11-03T07:18:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.