Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulation

• URL: http://arxiv.org/abs/2411.12820v1
• Date: Tue, 19 Nov 2024 19:13:56 GMT
• Title: Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulation
• Authors: Peter Barnett, Lisa Thiergart,
• Abstract summary: We argue that regulation should require developers to explicitly identify and justify key underlying assumptions about evaluations. We identify core assumptions in AI evaluations, such as comprehensive threat modeling, proxy task validity, and adequate capability elicitation. Our presented approach aims to enhance transparency in AI development, offering a practical path towards more effective governance of advanced AI systems.
• Score: 2.07180164747172
• License:
• Abstract: As AI systems advance, AI evaluations are becoming an important pillar of regulations for ensuring safety. We argue that such regulation should require developers to explicitly identify and justify key underlying assumptions about evaluations as part of their case for safety. We identify core assumptions in AI evaluations (both for evaluating existing models and forecasting future models), such as comprehensive threat modeling, proxy task validity, and adequate capability elicitation. Many of these assumptions cannot currently be well justified. If regulation is to be based on evaluations, it should require that AI development be halted if evaluations demonstrate unacceptable danger or if these assumptions are inadequately justified. Our presented approach aims to enhance transparency in AI development, offering a practical path towards more effective governance of advanced AI systems.

Related papers

• Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization [53.80919781981027]
  Key requirements for trustworthy AI can be translated into design choices for the components of empirical risk minimization. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.
  arXiv  Detail & Related papers  (2024-10-25T07:53:32Z)
• How Could Generative AI Support Compliance with the EU AI Act? A Review for Safe Automated Driving Perception [4.075971633195745]
  Deep Neural Networks (DNNs) have become central for the perception functions of autonomous vehicles. The European Union (EU) Artificial Intelligence (AI) Act aims to address these challenges by establishing stringent norms and standards for AI systems. This review paper summarizes the requirements arising from the EU AI Act regarding DNN-based perception systems and systematically categorizes existing generative AI applications in AD.
  arXiv  Detail & Related papers  (2024-08-30T12:01:06Z)
• The Dilemma of Uncertainty Estimation for General Purpose AI in the EU AI Act [6.9060054915724]
  The AI act is the European Union-wide regulation of AI systems. We argue that uncertainty estimation should be a required component for deploying models in the real world.
  arXiv  Detail & Related papers  (2024-08-20T23:59:51Z)
• Evaluating AI Evaluation: Perils and Prospects [8.086002368038658]
  This paper contends that the prevalent evaluation methods for these systems are fundamentally inadequate. I argue that a reformation is required in the way we evaluate AI systems and that we should look towards cognitive sciences for inspiration.
  arXiv  Detail & Related papers  (2024-07-12T12:37:13Z)
• A Nested Model for AI Design and Validation [0.5120567378386615]
  Despite the need for new regulations, there is a mismatch between regulatory science and AI. A five-layer nested model for AI design and validation aims to address these issues.
  arXiv  Detail & Related papers  (2024-06-08T12:46:12Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Functional trustworthiness of AI systems by statistically valid testing [7.717286312400472]
  The authors are concerned about the safety, health, and rights of the European citizens due to inadequate measures and procedures required by the current draft of the EU Artificial Intelligence (AI) Act. We observe that not only the current draft of the EU AI Act, but also the accompanying standardization efforts in CEN/CENELEC, have resorted to the position that real functional guarantees of AI systems supposedly would be unrealistic and too complex anyways.
  arXiv  Detail & Related papers  (2023-10-04T11:07:52Z)
• Model evaluation for extreme risks [46.53170857607407]
  Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is critical for addressing extreme risks.
  arXiv  Detail & Related papers  (2023-05-24T16:38:43Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims [59.64274607533249]
  AI developers need to make verifiable claims to which they can be held accountable. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.
  arXiv  Detail & Related papers  (2020-04-15T17:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.