Black-Box Access is Insufficient for Rigorous AI Audits

• URL: http://arxiv.org/abs/2401.14446v3
• Date: Wed, 29 May 2024 13:56:29 GMT
• Title: Black-Box Access is Insufficient for Rigorous AI Audits
• Authors: Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin Von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau, Max Tegmark, David Krueger, Dylan Hadfield-Menell,
• Abstract summary: We discuss the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks.
• Score: 25.983796391987298
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depends on the degree of access granted to auditors. Recent audits of state-of-the-art AI systems have primarily relied on black-box access, in which auditors can only query the system and observe its outputs. However, white-box access to the system's inner workings (e.g., weights, activations, gradients) allows an auditor to perform stronger attacks, more thoroughly interpret models, and conduct fine-tuning. Meanwhile, outside-the-box access to training and deployment information (e.g., methodology, code, documentation, data, deployment details, findings from internal evaluations) allows auditors to scrutinize the development process and design more targeted evaluations. In this paper, we examine the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks. Given that different forms of access can lead to very different levels of evaluation, we conclude that (1) transparency regarding the access and methods used by auditors is necessary to properly interpret audit results, and (2) white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.

Related papers

• Auditing for Bias in Ad Delivery Using Inferred Demographic Attributes [50.37313459134418]
  We study the effects of inference error on auditing for bias in one prominent application: black-box audit of ad delivery using paired ads. We propose a way to mitigate the inference error when evaluating skew in ad delivery algorithms.
  arXiv  Detail & Related papers  (2024-10-30T18:57:03Z)
• From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing [1.196505602609637]
  Audits can take many forms, including pre-deployment risk assessments, ongoing monitoring, and compliance testing. There are many operational challenges to AI auditing that complicate its implementation. We argue that auditing can be cast as a natural hypothesis test, draw parallels hypothesis testing and legal procedure, and argue that this framing provides clear and interpretable guidance on audit implementation.
  arXiv  Detail & Related papers  (2024-10-07T06:15:46Z)
• Auditing of AI: Legal, Ethical and Technical Approaches [0.0]
  AI auditing is a rapidly growing field of research and practice. Different approaches to AI auditing have different affordances and constraints. The next step in the evolution of auditing as an AI governance mechanism should be the interlinking of these available approaches.
  arXiv  Detail & Related papers  (2024-07-07T12:49:58Z)
• A Game-Theoretic Analysis of Auditing Differentially Private Algorithms with Epistemically Disparate Herd [16.10098472773814]
  This study examines the impact of herd audits on algorithm developers using the Stackelberg game approach. By enhancing transparency and accountability, herd audit contributes to the responsible development of privacy-preserving algorithms.
  arXiv  Detail & Related papers  (2024-04-24T20:34:27Z)
• The Decisive Power of Indecision: Low-Variance Risk-Limiting Audits and Election Contestation via Marginal Mark Recording [51.82772358241505]
  Risk-limiting audits (RLAs) are techniques for verifying the outcomes of large elections. We define new families of audits that improve efficiency and offer advances in statistical power. New audits are enabled by revisiting the standard notion of a cast-vote record so that it can declare multiple possible mark interpretations.
  arXiv  Detail & Related papers  (2024-02-09T16:23:54Z)
• A Framework for Assurance Audits of Algorithmic Systems [2.2342503377379725]
  We propose the criterion audit as an operationalizable compliance and assurance external audit framework. We argue that AI audits should similarly provide assurance to their stakeholders about AI organizations' ability to govern their algorithms in ways that harms and uphold human values. We conclude by offering a critical discussion on the benefits, inherent limitations, and implementation challenges of applying practices of the more mature financial auditing industry to AI auditing.
  arXiv  Detail & Related papers  (2024-01-26T14:38:54Z)
• Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem [0.971392598996499]
  We provide the first comprehensive field scan of the AI audit ecosystem. We identify emerging best practices as well as methods and tools that are becoming commonplace. We outline policy recommendations to improve the quality and impact of these audits.
  arXiv  Detail & Related papers  (2023-10-04T01:40:03Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• FAT Forensics: A Python Toolbox for Implementing and Deploying Fairness, Accountability and Transparency Algorithms in Predictive Systems [69.24490096929709]
  We developed an open source Python package called FAT Forensics. It can inspect important fairness, accountability and transparency aspects of predictive algorithms. Our toolbox can evaluate all elements of a predictive pipeline.
  arXiv  Detail & Related papers  (2022-09-08T13:25:02Z)
• Having your Privacy Cake and Eating it Too: Platform-supported Auditing of Social Media Algorithms for Public Interest [70.02478301291264]
  Social media platforms curate access to information and opportunities, and so play a critical role in shaping public discourse. Prior studies have used black-box methods to show that these algorithms can lead to biased or discriminatory outcomes. We propose a new method for platform-supported auditing that can meet the goals of the proposed legislation.
  arXiv  Detail & Related papers  (2022-07-18T17:32:35Z)
• Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models [73.24381010980606]
  This study examines issues of algorithmic fairness in the context of systems that inform tax audit selection by the IRS. We show how the use of more flexible machine learning methods for selecting audits may affect vertical equity. Our results have implications for the design of algorithmic tools across the public sector.
  arXiv  Detail & Related papers  (2022-06-20T16:27:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.