Contemporary AI foundation models increase biological weapons risk

• URL: http://arxiv.org/abs/2506.13798v1
• Date: Thu, 12 Jun 2025 19:53:38 GMT
• Title: Contemporary AI foundation models increase biological weapons risk
• Abstract summary: We argue current foundation AI models underestimate the risk of facilitating biological weapons development.<n>We identify "elements of success" for biological weapons development that large language models can describe in words.<n>We advocate for improved benchmarks, while acknowledging the window for meaningful implementation may have already closed.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The rapid advancement of artificial intelligence has raised concerns about its potential to facilitate biological weapons development. We argue existing safety assessments of contemporary foundation AI models underestimate this risk, largely due to flawed assumptions and inadequate evaluation methods. First, assessments mistakenly assume biological weapons development requires tacit knowledge, or skills gained through hands-on experience that cannot be easily verbalized. Second, they rely on imperfect benchmarks that overlook how AI can uplift both nonexperts and already-skilled individuals. To challenge the tacit knowledge assumption, we examine cases where individuals without formal expertise, including a 2011 Norwegian ultranationalist who synthesized explosives, successfully carried out complex technical tasks. We also review efforts to document pathogen construction processes, highlighting how such tasks can be conveyed in text. We identify "elements of success" for biological weapons development that large language models can describe in words, including steps such as acquiring materials and performing technical procedures. Applying this framework, we find that advanced AI models Llama 3.1 405B, ChatGPT-4o, and Claude 3.5 Sonnet can accurately guide users through the recovery of live poliovirus from commercially obtained synthetic DNA, challenging recent claims that current models pose minimal biosecurity risk. We advocate for improved benchmarks, while acknowledging the window for meaningful implementation may have already closed.

Related papers

• Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery [81.03797680309154]
  DBench-Bio is a dynamic and fully automated benchmark to evaluate AI's biological knowledge discovery ability.<n>We instantiate this pipeline to construct a monthly-updated benchmark covering 12 biomedical sub-domains.<n>Our work provides the first dynamic, automatic framework for assessing the new knowledge discovery capabilities of AI systems.
  arXiv  Detail & Related papers  (2026-02-10T05:47:22Z)
• Toward Risk Thresholds for AI-Enabled Cyber Threats: Enhancing Decision-Making Under Uncertainty with Bayesian Networks [0.3151064009829256]
  We propose a structured approach to developing and evaluating AI cyber risk thresholds.<n>First, we analyze existing industry cyber thresholds and identify common threshold elements.<n>Second, we propose the use of Bayesian networks as a tool for modeling AI-enabled cyber risk.
  arXiv  Detail & Related papers  (2026-01-23T23:23:12Z)
• Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse [50.87630846876635]
  We develop nine detailed cyber risk models.<n>Each model decomposes attacks into steps using the MITRE ATT&CK framework.<n>Individual estimates are aggregated through Monte Carlo simulation.
  arXiv  Detail & Related papers  (2025-12-09T17:54:17Z)
• Adversarially-Aware Architecture Design for Robust Medical AI Systems [0.0]
  Adversarial attacks pose a severe risk to AI systems used in healthcare.<n>Our study explores these vulnerabilities through empirical experimentation on a dermatological dataset.<n>We conclude with a call for integrated technical, ethical, and policy-based approaches to build more resilient, equitable AI in healthcare.
  arXiv  Detail & Related papers  (2025-10-23T16:51:11Z)
• Deep Learning Models for Robust Facial Liveness Detection [56.08694048252482]
  This study introduces a robust solution through novel deep learning models addressing the deficiencies in contemporary anti-spoofing techniques.<n>By innovatively integrating texture analysis and reflective properties associated with genuine human traits, our models distinguish authentic presence from replicas with remarkable precision.
  arXiv  Detail & Related papers  (2025-08-12T17:19:20Z)
• A Proposal for Evaluating the Operational Risk for ChatBots based on Large Language Models [39.58317527488534]
  We propose a novel, instrumented risk-assessment metric that simultaneously evaluates potential threats to three key stakeholders.<n>To validate our metric, we leverage Garak, an open-source framework for vulnerability testing.<n>Results underscore the importance of multi-dimensional risk assessments in operationalizing secure, reliable AI-driven conversational systems.
  arXiv  Detail & Related papers  (2025-05-07T20:26:45Z)
• Can AI Agents Design and Implement Drug Discovery Pipelines? [1.5848629658789695]
  Current AI agent-based systems demonstrate proficiency in solving programming challenges and conducting research.<n>This paper introduces DO Challenge, a benchmark designed to evaluate the decision-making abilities of AI agents.<n>We present the Deep Thought multi-agent system, which demonstrated strong performance on the benchmark, outperforming most human teams.
  arXiv  Detail & Related papers  (2025-04-28T15:41:28Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Computational Safety for Generative AI: A Signal Processing Perspective [65.268245109828]
  computational safety is a mathematical framework that enables the quantitative assessment, formulation, and study of safety challenges in GenAI.<n>We show how sensitivity analysis and loss landscape analysis can be used to detect malicious prompts with jailbreak attempts.<n>We discuss key open research challenges, opportunities, and the essential role of signal processing in computational AI safety.
  arXiv  Detail & Related papers  (2025-02-18T02:26:50Z)
• The Elicitation Game: Evaluating Capability Elicitation Techniques [1.064108398661507]
  We evaluate the effectiveness of capability elicitation techniques by intentionally training model organisms.<n>We introduce a novel method for training model organisms, based on circuit breaking.<n>For a code-generation task, only fine-tuning can elicit the hidden capabilities of our novel model organism.
  arXiv  Detail & Related papers  (2025-02-04T09:54:24Z)
• Safety at Scale: A Comprehensive Survey of Large Model Safety [298.05093528230753]
  We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats.<n>We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
  arXiv  Detail & Related papers  (2025-02-02T05:14:22Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Generative AI for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations [12.73011921253]
  This review introduces the transformative potential of generative Artificial Intelligence (AI) and foundation models, including large language models (LLMs), for health technology assessment (HTA) We explore their applications in four critical areas, synthesis evidence, evidence generation, clinical trials and economic modeling. Despite their promise, these technologies, while rapidly improving, are still nascent and continued careful evaluation in their applications to HTA is required.
  arXiv  Detail & Related papers  (2024-07-09T09:25:27Z)
• Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Models [0.0]
  We argue that AI evaluations model should prioritize addressing high-consequence risks. These risks could cause large-scale harm to the public, such as pandemics. Scientists' experience with identifying and mitigating dual-use biological risks can help inform new approaches to evaluating biological AI models.
  arXiv  Detail & Related papers  (2024-05-25T16:29:17Z)
• Evaluating Frontier Models for Dangerous Capabilities [59.129424649740855]
  We introduce a programme of "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.
  arXiv  Detail & Related papers  (2024-03-20T17:54:26Z)
• Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy [65.77763092833348]
  This perspective examines vulnerabilities in AI scientists, shedding light on potential risks associated with their misuse.<n>We take into account user intent, the specific scientific domain, and their potential impact on the external environment.<n>We propose a triadic framework involving human regulation, agent alignment, and an understanding of environmental feedback.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.