An International Agreement to Prevent the Premature Creation of Artificial Superintelligence

• URL: http://arxiv.org/abs/2511.10783v2
• Date: Mon, 17 Nov 2025 02:16:23 GMT
• Title: An International Agreement to Prevent the Premature Creation of Artificial Superintelligence
• Authors: Aaron Scher, David Abecassis, Peter Barnett, Brian Abeyta,
• Abstract summary: This report proposes an international agreement to prevent the premature development of artificial superintelligence.<n>The agreement halts dangerous AI capabilities advancement while preserving access to current, safe AI applications.<n>Due to the lack of trust between parties, verification is a key part of the agreement.
• Score: 1.5749416770494706
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many experts argue that premature development of artificial superintelligence (ASI) poses catastrophic risks, including the risk of human extinction from misaligned ASI, geopolitical instability, and misuse by malicious actors. This report proposes an international agreement to prevent the premature development of ASI until AI development can proceed without these risks. The agreement halts dangerous AI capabilities advancement while preserving access to current, safe AI applications. The proposed framework centers on a coalition led by the United States and China that would restrict the scale of AI training and dangerous AI research. Due to the lack of trust between parties, verification is a key part of the agreement. Limits on the scale of AI training are operationalized by FLOP thresholds and verified through the tracking of AI chips and verification of chip use. Dangerous AI research--that which advances toward artificial superintelligence or endangers the agreement's verifiability--is stopped via legal prohibitions and multifaceted verification. We believe the proposal would be technically sufficient to forestall the development of ASI if implemented today, but advancements in AI capabilities or development methods could hurt its efficacy. Additionally, there does not yet exist the political will to put such an agreement in place. Despite these challenges, we hope this agreement can provide direction for AI governance research and policy.

Related papers

• "We are not Future-ready": Understanding AI Privacy Risks and Existing Mitigation Strategies from the Perspective of AI Developers in Europe [56.1653658714305]
  We interviewed 25 AI developers based in Europe to understand which privacy threats they believe pose the greatest risk to users, developers, and businesses.<n>We find that there is little consensus among AI developers on the relative ranking of privacy risks.<n>While AI developers are aware of proposed mitigation strategies for addressing these risks, they reported minimal real-world adoption.
  arXiv  Detail & Related papers  (2025-10-01T13:51:33Z)
• Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment [0.7364983833280243]
  This report provides an in-depth overview of AI verification, intended for both policy professionals and technical researchers.<n>We present novel conceptual frameworks, detailed implementation options, and key R&D challenges.<n>We find that states could eventually verify compliance by using six largely independent verification approaches.
  arXiv  Detail & Related papers  (2025-07-21T17:45:15Z)
• Domestic frontier AI regulation, an IAEA for AI, an NPT for AI, and a US-led Allied Public-Private Partnership for AI: Four institutions for governing and developing frontier AI [0.0]
  I explore four institutions for governing and developing frontier AI.<n>Domestic regimes could be harmonized and monitored through an IAEA for AI.<n>This could be backed up by a Secure Chips Agreement - a Non-Proliferation Treaty (NPT) for AI.<n> Frontier training runs could be carried out by a megaproject between the USA and its allies.
  arXiv  Detail & Related papers  (2025-07-08T20:32:28Z)
• The Singapore Consensus on Global AI Safety Research Priorities [128.58674892183657]
  "2025 Singapore Conference on AI (SCAI): International Scientific Exchange on AI Safety" aimed to support research in this space.<n>Report builds on the International AI Safety Report chaired by Yoshua Bengio and backed by 33 governments.<n>Report organises AI safety research domains into three types: challenges with creating trustworthy AI systems (Development), challenges with evaluating their risks (Assessment) and challenges with monitoring and intervening after deployment (Control)
  arXiv  Detail & Related papers  (2025-06-25T17:59:50Z)
• Mechanisms to Verify International Agreements About AI Development [0.0]
  Report aims to demonstrate how countries could practically verify claims about each other's AI development and deployment.<n>The focus is on international agreements and state-involved AI development, but these approaches could also be applied to domestic regulation of companies.
  arXiv  Detail & Related papers  (2025-06-18T20:28:54Z)
• The California Report on Frontier AI Policy [110.35302787349856]
  Continued progress in frontier AI carries the potential for profound advances in scientific discovery, economic productivity, and broader social well-being.<n>As the epicenter of global AI innovation, California has a unique opportunity to continue supporting developments in frontier AI.<n>Report derives policy principles that can inform how California approaches the use, assessment, and governance of frontier AI.
  arXiv  Detail & Related papers  (2025-06-17T23:33:21Z)
• Superintelligence Strategy: Expert Version [64.7113737051525]
  Destabilizing AI developments could raise the odds of great-power conflict.<n>Superintelligence -- AI vastly better than humans at nearly all cognitive tasks -- is now anticipated by AI researchers.<n>We introduce the concept of Mutual Assured AI Malfunction.
  arXiv  Detail & Related papers  (2025-03-07T17:53:24Z)
• Position: Mind the Gap-the Growing Disconnect Between Established Vulnerability Disclosure and AI Security [56.219994752894294]
  We argue that adapting existing processes for AI security reporting is doomed to fail due to fundamental shortcomings for the distinctive characteristics of AI systems.<n>Based on our proposal to address these shortcomings, we discuss an approach to AI security reporting and how the new AI paradigm, AI agents, will further reinforce the need for specialized AI security incident reporting advancements.
  arXiv  Detail & Related papers  (2024-12-19T13:50:26Z)
• Using AI Alignment Theory to understand the potential pitfalls of regulatory frameworks [55.2480439325792]
  This paper critically examines the European Union's Artificial Intelligence Act (EU AI Act) Uses insights from Alignment Theory (AT) research, which focuses on the potential pitfalls of technical alignment in Artificial Intelligence. As we apply these concepts to the EU AI Act, we uncover potential vulnerabilities and areas for improvement in the regulation.
  arXiv  Detail & Related papers  (2024-10-10T17:38:38Z)
• Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
  General purpose AI seems to have lowered the barriers for the public to use AI and harness its power. We introduce PARTICIP-AI, a framework for laypeople to speculate and assess AI use cases and their impacts.
  arXiv  Detail & Related papers  (2024-03-21T19:12:37Z)
• Taking control: Policies to address extinction risks from advanced AI [0.0]
  We argue that voluntary commitments from AI companies would be an inappropriate and insufficient response.<n>We describe three policy proposals that would meaningfully address the threats from advanced AI.
  arXiv  Detail & Related papers  (2023-10-31T15:53:14Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Nuclear Arms Control Verification and Lessons for AI Treaties [0.0]
  Security risks from AI have motivated international agreements that the technology can be used. The study suggests that the foreseeable case would be reduced to levels that were successfully managed in nuclear arms control.
  arXiv  Detail & Related papers  (2023-04-08T23:05:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.