AuditMAI: Towards An Infrastructure for Continuous AI Auditing

• URL: http://arxiv.org/abs/2406.14243v1
• Date: Thu, 20 Jun 2024 12:11:53 GMT
• Title: AuditMAI: Towards An Infrastructure for Continuous AI Auditing
• Authors: Laura Waltersdorfer, Fajar J. Ekaputra, Tomasz Miksa, Marta Sabou,
• Abstract summary: Auditability is a core requirement for achieving responsible AI system design. Existing AI auditing tools typically lack integration features and remain as isolated approaches. Inspired by other domains such as finance, continuous AI auditing is a promising direction to conduct regular assessments of AI systems.
• Score: 0.6749750044497732
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial Intelligence (AI) Auditability is a core requirement for achieving responsible AI system design. However, it is not yet a prominent design feature in current applications. Existing AI auditing tools typically lack integration features and remain as isolated approaches. This results in manual, high-effort, and mostly one-off AI audits, necessitating alternative methods. Inspired by other domains such as finance, continuous AI auditing is a promising direction to conduct regular assessments of AI systems. The issue remains, however, since the methods for continuous AI auditing are not mature yet at the moment. To address this gap, we propose the Auditability Method for AI (AuditMAI), which is intended as a blueprint for an infrastructure towards continuous AI auditing. For this purpose, we first clarified the definition of AI auditability based on literature. Secondly, we derived requirements from two industrial use cases for continuous AI auditing tool support. Finally, we developed AuditMAI and discussed its elements as a blueprint for a continuous AI auditability infrastructure.

Related papers

• Using AI Alignment Theory to understand the potential pitfalls of regulatory frameworks [55.2480439325792]
  This paper critically examines the European Union's Artificial Intelligence Act (EU AI Act) Uses insights from Alignment Theory (AT) research, which focuses on the potential pitfalls of technical alignment in Artificial Intelligence. As we apply these concepts to the EU AI Act, we uncover potential vulnerabilities and areas for improvement in the regulation.
  arXiv  Detail & Related papers  (2024-10-10T17:38:38Z)
• Auditing of AI: Legal, Ethical and Technical Approaches [0.0]
  AI auditing is a rapidly growing field of research and practice. Different approaches to AI auditing have different affordances and constraints. The next step in the evolution of auditing as an AI governance mechanism should be the interlinking of these available approaches.
  arXiv  Detail & Related papers  (2024-07-07T12:49:58Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• The Necessity of AI Audit Standards Boards [0.0]
  We argue that creating auditing standards is not just insufficient, but actively harmful by proliferating unheeded and inconsistent standards. Instead, the paper proposes the establishment of an AI Audit Standards Board, responsible for developing and updating auditing methods and standards.
  arXiv  Detail & Related papers  (2024-04-11T15:08:24Z)
• Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling [1.841662059101602]
  Audits are critical mechanisms for identifying the risks and limitations of deployed artificial intelligence (AI) systems. We map the current ecosystem of available AI audit tools. We conclude that resources are lacking to adequately support the full scope of needs for many AI audit practitioners.
  arXiv  Detail & Related papers  (2024-02-27T19:52:54Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Advancing AI Audits for Enhanced AI Governance [1.875782323187985]
  This policy recommendation summarizes the issues related to the auditing of AI services and systems. It presents three recommendations for promoting AI auditing that contribute to sound AI governance.
  arXiv  Detail & Related papers  (2023-11-26T16:18:17Z)
• AI Maintenance: A Robustness Perspective [91.28724422822003]
  We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance. We propose an AI model inspection framework to detect and mitigate robustness risks. Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
  arXiv  Detail & Related papers  (2023-01-08T15:02:38Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• The application of artificial intelligence in software engineering: a review challenging conventional wisdom [0.9651131604396904]
  This survey chapter is a review of the most commonplace methods of AI applied to software engineering. The review covers methods between years 1975-2017, for the requirements phase, 46 major AI-driven methods are found. The purpose of this chapter is to answer the following questions: is there sufficient intelligence in the SE lifecycle?
  arXiv  Detail & Related papers  (2021-08-03T15:59:59Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.