Monte Carlo Expected Threat (MOCET) Scoring

• URL: http://arxiv.org/abs/2511.16823v1
• Date: Thu, 20 Nov 2025 22:06:13 GMT
• Title: Monte Carlo Expected Threat (MOCET) Scoring
• Authors: Joseph Kim, Saahith Potluri,
• Abstract summary: ASL-3+ models present a unique risk in their ability to uplift novice non-state actors.<n>Existing evaluation metrics can reliably assess model uplift and domain knowledge.<n>We introduce MOCET, an interpretable and doubly-scalable metric that can quantify real-world risks.
• Score: 1.4216140193392368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evaluating and measuring AI Safety Level (ASL) threats are crucial for guiding stakeholders to implement safeguards that keep risks within acceptable limits. ASL-3+ models present a unique risk in their ability to uplift novice non-state actors, especially in the realm of biosecurity. Existing evaluation metrics, such as LAB-Bench, BioLP-bench, and WMDP, can reliably assess model uplift and domain knowledge. However, metrics that better contextualize "real-world risks" are needed to inform the safety case for LLMs, along with scalable, open-ended metrics to keep pace with their rapid advancements. To address both gaps, we introduce MOCET, an interpretable and doubly-scalable metric (automatable and open-ended) that can quantify real-world risks.

Related papers

• Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential [0.1631115063641726]
  This research provides an empirical analysis and risk profile of nine prominent Large Language Models (LLMs)<n>The RSI is an agile and scalable evaluation score, to quantify and compare the security posture and creating a risk profile of LLMs.<n>This research finds widespread vulnerabilities in the safety filters of the LLMs tested and highlights the urgent need for stronger alignment, responsible deployment practices, and model governance.
  arXiv  Detail & Related papers  (2025-09-12T19:34:10Z)
• AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions [64.85086226439954]
  We present SAFE, a benchmark for assessing the safety of embodied VLM agents on hazardous instructions.<n> SAFE comprises three components: SAFE-THOR, SAFE-VERSE, and SAFE-DIAGNOSE.<n>We uncover systematic failures in translating hazard recognition into safe planning and execution.
  arXiv  Detail & Related papers  (2025-06-17T16:37:35Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• Mapping AI Benchmark Data to Quantitative Risk Estimates Through Expert Elicitation [0.7889270818022226]
  We show how existing AI benchmarks can be used to facilitate the creation of risk estimates.<n>We describe the results of a pilot study in which experts use information from Cybench, an AI benchmark, to generate probability estimates.
  arXiv  Detail & Related papers  (2025-03-06T10:39:47Z)
• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models [75.67623347512368]
  We propose toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs.
  arXiv  Detail & Related papers  (2024-10-24T17:14:40Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Current state of LLM Risks and AI Guardrails [0.0]
  Large language models (LLMs) have become increasingly sophisticated, leading to widespread deployment in sensitive applications where safety and reliability are paramount. These risks necessitate the development of "guardrails" to align LLMs with desired behaviors and mitigate potential harm. This work explores the risks associated with deploying LLMs and evaluates current approaches to implementing guardrails and model alignment techniques.
  arXiv  Detail & Related papers  (2024-06-16T22:04:10Z)
• S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models [46.148439517272024]
  Generative large language models (LLMs) have revolutionized natural language processing with their transformative and emergent capabilities.<n>Recent evidence indicates that LLMs can produce harmful content that violates social norms.<n>We propose S-Eval, an automated Safety Evaluation framework with a newly defined comprehensive risk taxonomy.
  arXiv  Detail & Related papers  (2024-05-23T05:34:31Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal [0.0]
  We propose a risk assessment process using tools like the risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. We also map threats against three key stakeholder groups.
  arXiv  Detail & Related papers  (2024-03-20T05:17:22Z)
• Safety Assessment of Chinese Large Language Models [51.83369778259149]
  Large language models (LLMs) may generate insulting and discriminatory content, reflect incorrect social values, and may be used for malicious purposes. To promote the deployment of safe, responsible, and ethical AI, we release SafetyPrompts including 100k augmented prompts and responses by LLMs.
  arXiv  Detail & Related papers  (2023-04-20T16:27:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.