Understanding the First Wave of AI Safety Institutes: Characteristics, Functions, and Challenges

• URL: http://arxiv.org/abs/2410.09219v1
• Date: Fri, 11 Oct 2024 19:50:23 GMT
• Title: Understanding the First Wave of AI Safety Institutes: Characteristics, Functions, and Challenges
• Authors: Renan Araujo, Kristina Fort, Oliver Guest,
• Abstract summary: In November 2023, the UK and US announced the creation of their AI Safety Institutes. This primer describes one cluster of similar clusters, the "first wave" First-waves have several fundamental characteristics in common.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In November 2023, the UK and US announced the creation of their AI Safety Institutes (AISIs). Five other jurisdictions have followed in establishing AISIs or similar institutions, with more likely to follow. While there is considerable variation between these institutions, there are also key similarities worth identifying. This primer describes one cluster of similar AISIs, the "first wave," consisting of the Japan, UK, and US AISIs. First-wave AISIs have several fundamental characteristics in common: they are technical government institutions, have a clear mandate related to the safety of advanced AI systems, and lack regulatory powers. Safety evaluations are at the center of first-wave AISIs. These techniques test AI systems across tasks to understand their behavior and capabilities on relevant risks, such as cyber, chemical, and biological misuse. They also share three core functions: research, standards, and cooperation. These functions are critical to AISIs' work on safety evaluations but also support other activities such as scientific consensus-building and foundational AI safety research. Despite its growing popularity as an institutional model, the AISI model is not free from challenges and limitations. Some analysts have criticized the first wave of AISIs for specializing too much in a sub-area and for being potentially redundant with existing institutions, for example. Future developments may rapidly change this landscape, and particularities of individual AISIs may not be captured by our broad-strokes description. This policy brief aims to outline the core elements of first-wave AISIs as a way of encouraging and improving conversations on this novel institutional model, acknowledging this is just a simplified snapshot rather than a timeless prescription.

Related papers

• AI threats to national security can be countered through an incident regime [55.2480439325792]
  We propose a legally mandated post-deployment AI incident regime that aims to counter potential national security threats from AI systems. Our proposed AI incident regime is split into three phases. The first phase revolves around a novel operationalization of what counts as an 'AI incident' The second and third phases spell out that AI providers should notify a government agency about incidents, and that the government agency should be involved in amending AI providers' security and safety procedures.
  arXiv  Detail & Related papers  (2025-03-25T17:51:50Z)
• Which Information should the UK and US AISI share with an International Network of AISIs? Opportunities, Risks, and a Tentative Proposal [0.0]
  The UK AI Safety Institute (UK) and its parallel organisation in the United States (US) take up a unique position in the recently established International Network of jurisdictions. This paper argues that it is in the interest of both institutions to share specific categories of information with the International Network of evaluations.
  arXiv  Detail & Related papers  (2025-02-05T16:49:02Z)
• Using AI Alignment Theory to understand the potential pitfalls of regulatory frameworks [55.2480439325792]
  This paper critically examines the European Union's Artificial Intelligence Act (EU AI Act) Uses insights from Alignment Theory (AT) research, which focuses on the potential pitfalls of technical alignment in Artificial Intelligence. As we apply these concepts to the EU AI Act, we uncover potential vulnerabilities and areas for improvement in the regulation.
  arXiv  Detail & Related papers  (2024-10-10T17:38:38Z)
• Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI [52.138044013005]
  generative AI, particularly large language models (LLMs), become increasingly integrated into production applications. New attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems.
  arXiv  Detail & Related papers  (2024-09-23T10:18:10Z)
• The Role of AI Safety Institutes in Contributing to International Standards for Frontier AI Safety [0.0]
  We argue that the AI Safety Institutes (AISIs) are well-positioned to contribute to the international standard-setting processes for AI safety. We propose and evaluate three models for involvement: Seoul Declaration Signatories, US (and other Seoul Declaration Signatories) and China, and Globally Inclusive.
  arXiv  Detail & Related papers  (2024-09-17T16:12:54Z)
• AI Horizon Scanning, White Paper p3395, IEEE-SA. Part I: Areas of Attention [0.22470290096767004]
  This manuscript is the first of a series of White Papers informing the development of IEEE-SA's p3995: Standard for the Implementation of Safeguards, Controls, and Preventive Techniques for Artificial Intelligence (AI) Models' In this first horizon-scanning we identify key attention areas for standards activities in AI. We examine different principles for regulatory efforts, and review notions of accountability, privacy, data rights and mis-use.
  arXiv  Detail & Related papers  (2024-09-13T18:00:01Z)
• The potential functions of an international institution for AI safety. Insights from adjacent policy areas and recent trends [0.0]
  The OECD, the G7, the G20, UNESCO, and the Council of Europe have already started developing frameworks for ethical and responsible AI governance. This chapter reflects on what functions an international AI safety institute could perform.
  arXiv  Detail & Related papers  (2024-08-31T10:04:53Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context. We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Towards a Privacy and Security-Aware Framework for Ethical AI: Guiding the Development and Assessment of AI Systems [0.0]
  This study conducts a systematic literature review spanning the years 2020 to 2023. Through the synthesis of knowledge extracted from the SLR, this study presents a conceptual framework tailored for privacy- and security-aware AI systems.
  arXiv  Detail & Related papers  (2024-03-13T15:39:57Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI. It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation [61.77881142275982]
  This interdisciplinary position paper considers various concerns surrounding fairness and discrimination in AI, and discusses how AI regulations address them. We first look at AI and fairness through the lenses of law, (AI) industry, sociotechnology, and (moral) philosophy, and present various perspectives. We identify and propose the roles AI Regulation should take to make the endeavor of the AI Act a success in terms of AI fairness concerns.
  arXiv  Detail & Related papers  (2022-06-08T12:32:08Z)
• Hard Choices in Artificial Intelligence [0.8594140167290096]
  We show how this vagueness cannot be resolved through mathematical formalism alone. We show how this vagueness cannot be resolved through mathematical formalism alone.
  arXiv  Detail & Related papers  (2021-06-10T09:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.