In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?

• URL: http://arxiv.org/abs/2504.12914v1
• Date: Thu, 17 Apr 2025 13:03:56 GMT
• Title: In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
• Authors: Ben Bucknall, Saad Siddiqui, Lara Thurnherr, Conor McGurk, Ben Harack, Anka Reuel, Patricia Paskov, Casey Mahoney, Sören Mindermann, Scott Singer, Vinay Hiremath, Charbel-Raphaël Segerie, Oscar Delaney, Alessandro Abate, Fazl Barez, Michael K. Cohen, Philip Torr, Ferenc Huszár, Anisoara Calinescu, Gabriel Davis Jones, Yoshua Bengio, Robert Trager,
• Abstract summary: We consider technical factors that impact the risks of international cooperation on AI safety research.<n>We focus on the degree to which such cooperation can advance dangerous capabilities, result in the sharing of sensitive information, or provide opportunities for harm.<n>We argue that existing frameworks for managing associated risks can be supplemented with consideration of key risks specific to cooperation on technical AI safety research.
• Score: 66.89036079974998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: International cooperation is common in AI research, including between geopolitical rivals. While many experts advocate for greater international cooperation on AI safety to address shared global risks, some view cooperation on AI with suspicion, arguing that it can pose unacceptable risks to national security. However, the extent to which cooperation on AI safety poses such risks, as well as provides benefits, depends on the specific area of cooperation. In this paper, we consider technical factors that impact the risks of international cooperation on AI safety research, focusing on the degree to which such cooperation can advance dangerous capabilities, result in the sharing of sensitive information, or provide opportunities for harm. We begin by why nations historically cooperate on strategic technologies and analyse current US-China cooperation in AI as a case study. We further argue that existing frameworks for managing associated risks can be supplemented with consideration of key risks specific to cooperation on technical AI safety research. Through our analysis, we find that research into AI verification mechanisms and shared protocols may be suitable areas for such cooperation. Through this analysis we aim to help researchers and governments identify and mitigate the risks of international cooperation on AI safety research, so that the benefits of cooperation can be fully realised.

Related papers

• AI Safety for Everyone [3.440579243843689]
  Recent discussions and research in AI safety have increasingly emphasized the deep connection between AI safety and existential risk from advanced AI systems.<n>This framing may exclude researchers and practitioners who are committed to AI safety but approach the field from different angles.<n>We find a vast array of concrete safety work that addresses immediate and practical concerns with current AI systems.
  arXiv  Detail & Related papers  (2025-02-13T13:04:59Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context.<n>We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• US-China perspectives on extreme AI risks and global governance [0.0]
  We sought to better understand how experts in each country describe safety and security threats from advanced artificial intelligence. We focused our analysis on advanced forms of artificial intelligence, such as artificial general intelligence (AGI) Experts in both countries expressed concern about risks from AGI, risks from intelligence explosions, and risks from AI systems that escape human control.
  arXiv  Detail & Related papers  (2024-06-23T17:31:27Z)
• AI Risk Management Should Incorporate Both Safety and Security [185.68738503122114]
  We argue that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security. We introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security.
  arXiv  Detail & Related papers  (2024-05-29T21:00:47Z)
• Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas [15.785674974107204]
  The study of cooperation within social dilemmas has long been a fundamental topic across various disciplines. Recent advancements in Artificial Intelligence have significantly reshaped this field. This survey examines three key areas at the intersection of AI and cooperation in social dilemmas.
  arXiv  Detail & Related papers  (2024-02-27T07:31:30Z)
• Control Risk for Potential Misuse of Artificial Intelligence in Science [85.91232985405554]
  We aim to raise awareness of the dangers of AI misuse in science. We highlight real-world examples of misuse in chemical science. We propose a system called SciGuard to control misuse risks for AI models in science.
  arXiv  Detail & Related papers  (2023-12-11T18:50:57Z)
• International Institutions for Advanced AI [47.449762587672986]
  International institutions may have an important role to play in ensuring advanced AI systems benefit humanity. This paper identifies a set of governance functions that could be performed at an international level to address these challenges. It groups these functions into four institutional models that exhibit internal synergies and have precedents in existing organizations.
  arXiv  Detail & Related papers  (2023-07-10T16:55:55Z)
• Trustworthy, responsible, ethical AI in manufacturing and supply chains: synthesis and emerging research questions [59.34177693293227]
  We explore the applicability of responsible, ethical, and trustworthy AI within the context of manufacturing. We then use a broadened adaptation of a machine learning lifecycle to discuss, through the use of illustrative examples, how each step may result in a given AI trustworthiness concern.
  arXiv  Detail & Related papers  (2023-05-19T10:43:06Z)
• Accumulating Risk Capital Through Investing in Cooperation [12.053132866404972]
  We show that the trade-off between safety and cooperation is not severe, and you can receive exponentially large returns through cooperation from a small amount of risk. We propose a method for training policies that targets this objective, Accumulating Risk Capital Through Investing in Cooperation (ARCTIC)
  arXiv  Detail & Related papers  (2021-01-25T18:41:45Z)
• Open Problems in Cooperative AI [21.303564222227727]
  Research aims to study the many aspects of the problems of cooperation and to innovate in AI to contribute to solving these problems. This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms.
  arXiv  Detail & Related papers  (2020-12-15T21:39:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.