Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Models

• URL: http://arxiv.org/abs/2407.13059v2
• Date: Tue, 23 Jul 2024 01:08:25 GMT
• Title: Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Models
• Authors: Jaspreet Pannu, Doni Bloomfield, Alex Zhu, Robert MacKnight, Gabe Gomes, Anita Cicero, Thomas V. Inglesby,
• Abstract summary: We argue that AI evaluations model should prioritize addressing high-consequence risks. These risks could cause large-scale harm to the public, such as pandemics. Scientists' experience with identifying and mitigating dual-use biological risks can help inform new approaches to evaluating biological AI models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As a result of rapidly accelerating AI capabilities, over the past year, national governments and multinational bodies have announced efforts to address safety, security and ethics issues related to AI models. One high priority among these efforts is the mitigation of misuse of AI models. Many biologists have for decades sought to reduce the risks of scientific research that could lead, through accident or misuse, to high-consequence disease outbreaks. Scientists have carefully considered what types of life sciences research have the potential for both benefit and risk (dual-use), especially as scientific advances have accelerated our ability to engineer organisms and create novel variants of pathogens. Here we describe how previous experience and study by scientists and policy professionals of dual-use capabilities in the life sciences can inform risk evaluations of AI models with biological capabilities. We argue that AI model evaluations should prioritize addressing high-consequence risks (those that could cause large-scale harm to the public, such as pandemics), and that these risks should be evaluated prior to model deployment so as to allow potential biosafety and/or biosecurity measures. Scientists' experience with identifying and mitigating dual-use biological risks can help inform new approaches to evaluating biological AI models. Identifying which AI capabilities post the greatest biosecurity and biosafety concerns is necessary in order to establish targeted AI safety evaluation methods, secure these tools against accident and misuse, and avoid impeding immense potential benefits.

Related papers

• A Trilogy of AI Safety Frameworks: Paths from Facts and Knowledge Gaps to Reliable Predictions and New Knowledge [0.0]
  AI Safety has become a vital front-line concern of many scientists within and outside the AI community. There are many immediate and long term anticipated risks that range from existential risk to human existence to deep fakes and bias in machine learning systems.
  arXiv  Detail & Related papers  (2024-10-09T14:43:06Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context. We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• Evaluating Frontier Models for Dangerous Capabilities [59.129424649740855]
  We introduce a programme of "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.
  arXiv  Detail & Related papers  (2024-03-20T17:54:26Z)
• Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science [65.77763092833348]
  Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these agents also introduce novel vulnerabilities that demand careful consideration for safety. This paper conducts a thorough examination of vulnerabilities in LLM-based agents within scientific domains, shedding light on potential risks associated with their misuse and emphasizing the need for safety measures.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)
• The whack-a-mole governance challenge for AI-enabled synthetic biology: literature review and emerging frameworks [0.0]
  AI-enabled synthetic biology has tremendous potential but also significantly increases biorisks. How to achieve early warning systems that enable prevention and mitigation of future AI-enabled biohazards from the lab will constantly need to evolve. Recent advances in chatbots enabled by generative AI have revived fears that advanced biological insight can more easily get into the hands of malignant individuals or organizations.
  arXiv  Detail & Related papers  (2024-02-01T03:53:13Z)
• Towards Risk Analysis of the Impact of AI on the Deliberate Biological Threat Landscape [0.0]
  The perception that the convergence of biological engineering and artificial intelligence could enable increased biorisk has drawn attention to the governance of biotechnology and artificial intelligence. The 2023 Executive Order, Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, requires an assessment of how artificial intelligence can increase biorisk. The perspective concludes by noting that assessment and evaluation methodologies must keep pace with advances of AI in the life sciences.
  arXiv  Detail & Related papers  (2024-01-23T13:35:16Z)
• Control Risk for Potential Misuse of Artificial Intelligence in Science [85.91232985405554]
  We aim to raise awareness of the dangers of AI misuse in science. We highlight real-world examples of misuse in chemical science. We propose a system called SciGuard to control misuse risks for AI models in science.
  arXiv  Detail & Related papers  (2023-12-11T18:50:57Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence [67.70415658080121]
  Recent advances in machine learning and AI are disrupting technological innovation, product development, and society as a whole. AI has contributed less to fundamental science in part because large data sets of high-quality data for scientific practice and model discovery are more difficult to access. Here we explore and investigate aspects of an AI-driven, automated, closed-loop approach to scientific discovery.
  arXiv  Detail & Related papers  (2023-07-09T21:16:56Z)
• Quantitative AI Risk Assessments: Opportunities and Challenges [9.262092738841979]
  AI-based systems are increasingly being leveraged to provide value to organizations, individuals, and society. Risks have led to proposed regulations, litigation, and general societal concerns. This paper explores the concept of a quantitative AI Risk Assessment.
  arXiv  Detail & Related papers  (2022-09-13T21:47:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.