International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty

• URL: http://arxiv.org/abs/2503.18956v1
• Date: Tue, 18 Mar 2025 16:29:57 GMT
• Title: International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty
• Authors: Rebecca Scholefield, Samuel Martin, Otto Barten,
• Abstract summary: Malicious use or malfunction of advanced general-purpose AI (GPAI) poses risks that could lead to'marginalisation or extinction of humanity'<n>To address these risks, there are an increasing number of proposals for international agreements on AI safety.<n>We propose a treaty establishing a compute threshold above which development requires rigorous oversight.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The malicious use or malfunction of advanced general-purpose AI (GPAI) poses risks that, according to leading experts, could lead to the 'marginalisation or extinction of humanity.' To address these risks, there are an increasing number of proposals for international agreements on AI safety. In this paper, we review recent (2023-) proposals, identifying areas of consensus and disagreement, and drawing on related literature to assess their feasibility. We focus our discussion on risk thresholds, regulations, types of international agreement and five related processes: building scientific consensus, standardisation, auditing, verification and incentivisation. Based on this review, we propose a treaty establishing a compute threshold above which development requires rigorous oversight. This treaty would mandate complementary audits of models, information security and governance practices, overseen by an international network of AI Safety Institutes (AISIs) with authority to pause development if risks are unacceptable. Our approach combines immediately implementable measures with a flexible structure that can adapt to ongoing research.

Related papers

• Enhancing Trust Through Standards: A Comparative Risk-Impact Framework for Aligning ISO AI Standards with Global Ethical and Regulatory Contexts [0.0]
  ISO standards aim to foster responsible development by embedding fairness, transparency, and risk management into AI systems. Their effectiveness varies across diverse regulatory landscapes, from the EU's risk-based AI Act to China's stability-focused measures. This paper introduces a novel Comparative Risk-Impact Assessment Framework to evaluate how well ISO standards address ethical risks.
  arXiv  Detail & Related papers  (2025-04-22T00:44:20Z)
• AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations [5.984437476321095]
  frontier AI companies should report both pre- and post-mitigation safety evaluations. evaluating models at both stages provides policymakers with essential evidence to regulate deployment, access, and safety standards.
  arXiv  Detail & Related papers  (2025-03-17T17:56:43Z)
• Securing External Deeper-than-black-box GPAI Evaluations [49.1574468325115]
  This paper examines the critical challenges and potential solutions for conducting secure and effective external evaluations of general-purpose AI (GPAI) models. With the exponential growth in size, capability, reach and accompanying risk, ensuring accountability, safety, and public trust requires frameworks that go beyond traditional black-box methods.
  arXiv  Detail & Related papers  (2025-03-10T16:13:45Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.50078821423793]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Securing the AI Frontier: Urgent Ethical and Regulatory Imperatives for AI-Driven Cybersecurity [0.0]
  This paper critically examines the evolving ethical and regulatory challenges posed by the integration of artificial intelligence in cybersecurity.<n>We trace the historical development of AI regulation, highlighting major milestones from theoretical discussions in the 1940s to the implementation of recent global frameworks such as the European Union AI Act.<n>Ethical concerns such as bias, transparency, accountability, privacy, and human oversight are explored in depth, along with their implications for AI-driven cybersecurity systems.
  arXiv  Detail & Related papers  (2025-01-15T18:17:37Z)
• Advancing Trustworthy AI for Sustainable Development: Recommendations for Standardising AI Incident Reporting [2.209921757303168]
  The increasing use of AI technologies has led to increasing AI incidents, posing risks and causing harm to individuals, organizations, and society.<n>This study recognizes and addresses the lack of standardized protocols for reliably and comprehensively gathering such incident data.<n>It proposes nine actionable recommendations to enhance standardization efforts to address these gaps.
  arXiv  Detail & Related papers  (2025-01-01T17:34:57Z)
• Position: A taxonomy for reporting and describing AI security incidents [57.98317583163334]
  We argue that specific are required to describe and report security incidents of AI systems. Existing frameworks for either non-AI security or generic AI safety incident reporting are insufficient to capture the specific properties of AI security.
  arXiv  Detail & Related papers  (2024-12-19T13:50:26Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context.<n>We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• AI Risk Management Should Incorporate Both Safety and Security [185.68738503122114]
  We argue that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security. We introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security.
  arXiv  Detail & Related papers  (2024-05-29T21:00:47Z)
• The risks of risk-based AI regulation: taking liability seriously [46.90451304069951]
  The development and regulation of AI seems to have reached a critical stage. Some experts are calling for a moratorium on the training of AI systems more powerful than GPT-4. This paper analyses the most advanced legal proposal, the European Union's AI Act.
  arXiv  Detail & Related papers  (2023-11-03T12:51:37Z)
• An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI [10.550015825854837]
  A regulatory gap has permitted AI labs to conduct research, development, and deployment activities with minimal oversight. frontier AI system evaluations have been proposed as a way of assessing risks from the development and deployment of frontier AI systems. This paper proposes a solution in the form of an international consortium for AI risk evaluations, comprising both AI developers and third-party AI risk evaluators.
  arXiv  Detail & Related papers  (2023-10-22T23:37:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.