A Game-Theoretic Analysis of Auditing Differentially Private Algorithms with Epistemically Disparate Herd

• URL: http://arxiv.org/abs/2404.16195v1
• Date: Wed, 24 Apr 2024 20:34:27 GMT
• Title: A Game-Theoretic Analysis of Auditing Differentially Private Algorithms with Epistemically Disparate Herd
• Authors: Ya-Ting Yang, Tao Zhang, Quanyan Zhu,
• Abstract summary: This study examines the impact of herd audits on algorithm developers using the Stackelberg game approach. By enhancing transparency and accountability, herd audit contributes to the responsible development of privacy-preserving algorithms.
• Score: 16.10098472773814
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Privacy-preserving AI algorithms are widely adopted in various domains, but the lack of transparency might pose accountability issues. While auditing algorithms can address this issue, machine-based audit approaches are often costly and time-consuming. Herd audit, on the other hand, offers an alternative solution by harnessing collective intelligence. Nevertheless, the presence of epistemic disparity among auditors, resulting in varying levels of expertise and access to knowledge, may impact audit performance. An effective herd audit will establish a credible accountability threat for algorithm developers, incentivizing them to uphold their claims. In this study, our objective is to develop a systematic framework that examines the impact of herd audits on algorithm developers using the Stackelberg game approach. The optimal strategy for auditors emphasizes the importance of easy access to relevant information, as it increases the auditors' confidence in the audit process. Similarly, the optimal choice for developers suggests that herd audit is viable when auditors face lower costs in acquiring knowledge. By enhancing transparency and accountability, herd audit contributes to the responsible development of privacy-preserving algorithms.

Related papers

• From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing [1.196505602609637]
  Audits can take many forms, including pre-deployment risk assessments, ongoing monitoring, and compliance testing. There are many operational challenges to AI auditing that complicate its implementation. We argue that auditing can be cast as a natural hypothesis test, draw parallels hypothesis testing and legal procedure, and argue that this framing provides clear and interpretable guidance on audit implementation.
  arXiv  Detail & Related papers  (2024-10-07T06:15:46Z)
• The Decisive Power of Indecision: Low-Variance Risk-Limiting Audits and Election Contestation via Marginal Mark Recording [51.82772358241505]
  Risk-limiting audits (RLAs) are techniques for verifying the outcomes of large elections. We define new families of audits that improve efficiency and offer advances in statistical power. New audits are enabled by revisiting the standard notion of a cast-vote record so that it can declare multiple possible mark interpretations.
  arXiv  Detail & Related papers  (2024-02-09T16:23:54Z)
• A Framework for Assurance Audits of Algorithmic Systems [2.2342503377379725]
  We propose the criterion audit as an operationalizable compliance and assurance external audit framework. We argue that AI audits should similarly provide assurance to their stakeholders about AI organizations' ability to govern their algorithms in ways that harms and uphold human values. We conclude by offering a critical discussion on the benefits, inherent limitations, and implementation challenges of applying practices of the more mature financial auditing industry to AI auditing.
  arXiv  Detail & Related papers  (2024-01-26T14:38:54Z)
• Black-Box Access is Insufficient for Rigorous AI Audits [25.983796391987298]
  We discuss the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks.
  arXiv  Detail & Related papers  (2024-01-25T18:58:05Z)
• Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem [0.971392598996499]
  We provide the first comprehensive field scan of the AI audit ecosystem. We identify emerging best practices as well as methods and tools that are becoming commonplace. We outline policy recommendations to improve the quality and impact of these audits.
  arXiv  Detail & Related papers  (2023-10-04T01:40:03Z)
• The right to audit and power asymmetries in algorithm auditing [68.8204255655161]
  We elaborate on the challenges and asymmetries mentioned by Sandvig at the IC2S2 2021. We also contribute a discussion of the asymmetries that were not covered by Sandvig. We discuss the implications these asymmetries have for algorithm auditing research.
  arXiv  Detail & Related papers  (2023-02-16T13:57:41Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• Having your Privacy Cake and Eating it Too: Platform-supported Auditing of Social Media Algorithms for Public Interest [70.02478301291264]
  Social media platforms curate access to information and opportunities, and so play a critical role in shaping public discourse. Prior studies have used black-box methods to show that these algorithms can lead to biased or discriminatory outcomes. We propose a new method for platform-supported auditing that can meet the goals of the proposed legislation.
  arXiv  Detail & Related papers  (2022-07-18T17:32:35Z)
• Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models [73.24381010980606]
  This study examines issues of algorithmic fairness in the context of systems that inform tax audit selection by the IRS. We show how the use of more flexible machine learning methods for selecting audits may affect vertical equity. Our results have implications for the design of algorithmic tools across the public sector.
  arXiv  Detail & Related papers  (2022-06-20T16:27:06Z)
• Algorithmic audits of algorithms, and the law [3.9103337761169943]
  We focus on external audits that are conducted by interacting with the user side of the target algorithm. The legal framework in which these audits take place is mostly ambiguous to researchers developing them. This article highlights the relation of current audits with law, in order to structure the growing field of algorithm auditing.
  arXiv  Detail & Related papers  (2022-02-15T14:20:53Z)
• Confidence-Budget Matching for Sequential Budgeted Learning [69.77435313099366]
  We formalize decision-making problems with querying budget. We consider multi-armed bandits, linear bandits, and reinforcement learning problems. We show that CBM based algorithms perform well in the presence of adversity.
  arXiv  Detail & Related papers  (2021-02-05T19:56:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.