Safety cases for frontier AI

• URL: http://arxiv.org/abs/2410.21572v1
• Date: Mon, 28 Oct 2024 22:08:28 GMT
• Title: Safety cases for frontier AI
• Authors: Marie Davidsen Buhl, Gaurav Sett, Leonie Koessler, Jonas Schuett, Markus Anderljung,
• Abstract summary: Safety cases are reports that make a structured argument, supported by evidence, that a system is safe enough in a given operational context. Safety cases are already common in other safety-critical industries such as aviation and nuclear power. We explain why they may also be a useful tool in frontier AI governance, both in industry self-regulation and government regulation.
• Score: 0.8987776881291144
• License:
• Abstract: As frontier artificial intelligence (AI) systems become more capable, it becomes more important that developers can explain why their systems are sufficiently safe. One way to do so is via safety cases: reports that make a structured argument, supported by evidence, that a system is safe enough in a given operational context. Safety cases are already common in other safety-critical industries such as aviation and nuclear power. In this paper, we explain why they may also be a useful tool in frontier AI governance, both in industry self-regulation and government regulation. We then discuss the practicalities of safety cases, outlining how to produce a frontier AI safety case and discussing what still needs to happen before safety cases can substantially inform decisions.

Related papers

• Defining and Evaluating Physical Safety for Large Language Models [62.4971588282174]
  Large Language Models (LLMs) are increasingly used to control robotic systems such as drones. Their risks of causing physical threats and harm in real-world applications remain unexplored. We classify the physical safety risks of drones into four categories: (1) human-targeted threats, (2) object-targeted threats, (3) infrastructure attacks, and (4) regulatory violations.
  arXiv  Detail & Related papers  (2024-11-04T17:41:25Z)
• Towards evaluations-based safety cases for AI scheming [37.399946932069746]
  We propose three arguments that safety cases could use in relation to scheming. First, developers of frontier AI systems could argue that AI systems are not capable of scheming. Second, one could argue that AI systems are not capable of posing harm through scheming. Third, one could argue that control measures around the AI systems would prevent unacceptable outcomes even if the AI systems intentionally attempted to subvert them.
  arXiv  Detail & Related papers  (2024-10-29T17:55:29Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context. We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• Cross-Modality Safety Alignment [73.8765529028288]
  We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment. To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations. Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-21T16:14:15Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Affirmative safety: An approach to risk management for high-risk AI [6.133009503054252]
  We argue that entities developing or deploying high-risk AI systems should be required to present evidence of affirmative safety. We propose a risk management approach for advanced AI in which model developers must provide evidence that their activities keep certain risks below regulator-set thresholds.
  arXiv  Detail & Related papers  (2024-04-14T20:48:55Z)
• The Open Autonomy Safety Case Framework [3.2995359570845917]
  Safety cases have become a best practice for measuring, managing, and communicating the safety of autonomous vehicles. This paper introduces the Open Autonomy Safety Case Framework, developed over years of work with the autonomous vehicle industry.
  arXiv  Detail & Related papers  (2024-04-08T12:26:06Z)
• Safety Cases: How to Justify the Safety of Advanced AI Systems [5.097102520834254]
  As AI systems become more advanced, companies and regulators will make difficult decisions about whether it is safe to train and deploy them. We propose a framework for organizing a safety case and discuss four categories of arguments to justify safety. We evaluate concrete examples of arguments in each category and outline how arguments could be combined to justify that AI systems are safe to deploy.
  arXiv  Detail & Related papers  (2024-03-15T16:53:13Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• X-Risk Analysis for AI Research [24.78742908726579]
  We provide a guide for how to analyze AI x-risk. First, we review how systems can be made safer today. Next, we discuss strategies for having long-term impacts on the safety of future systems.
  arXiv  Detail & Related papers  (2022-06-13T00:22:50Z)
• Proceedings of the Artificial Intelligence for Cyber Security (AICS) Workshop at AAAI 2022 [55.573187938617636]
  The workshop will focus on the application of AI to problems in cyber security. Cyber systems generate large volumes of data, utilizing this effectively is beyond human capabilities.
  arXiv  Detail & Related papers  (2022-02-28T18:27:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.