Ensuring Fairness with Transparent Auditing of Quantitative Bias in AI Systems

• URL: http://arxiv.org/abs/2409.06708v1
• Date: Sat, 24 Aug 2024 17:16:50 GMT
• Title: Ensuring Fairness with Transparent Auditing of Quantitative Bias in AI Systems
• Authors: Chih-Cheng Rex Yuan, Bow-Yaw Wang,
• Abstract summary: AI systems may exhibit biases that lead decision-makers to draw unfair conclusions. We present a framework for auditing AI fairness involving third-party auditors and AI system providers. We have created a tool to facilitate systematic examination of AI systems.
• Score: 0.30693357740321775
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: With the rapid advancement of AI, there is a growing trend to integrate AI into decision-making processes. However, AI systems may exhibit biases that lead decision-makers to draw unfair conclusions. Notably, the COMPAS system used in the American justice system to evaluate recidivism was found to favor racial majority groups; specifically, it violates a fairness standard called equalized odds. Various measures have been proposed to assess AI fairness. We present a framework for auditing AI fairness, involving third-party auditors and AI system providers, and we have created a tool to facilitate systematic examination of AI systems. The tool is open-sourced and publicly available. Unlike traditional AI systems, we advocate a transparent white-box and statistics-based approach. It can be utilized by third-party auditors, AI developers, or the general public for reference when judging the fairness criterion of AI systems.

Related papers

• Using AI Alignment Theory to understand the potential pitfalls of regulatory frameworks [55.2480439325792]
  This paper critically examines the European Union's Artificial Intelligence Act (EU AI Act) Uses insights from Alignment Theory (AT) research, which focuses on the potential pitfalls of technical alignment in Artificial Intelligence. As we apply these concepts to the EU AI Act, we uncover potential vulnerabilities and areas for improvement in the regulation.
  arXiv  Detail & Related papers  (2024-10-10T17:38:38Z)
• Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
  General purpose AI seems to have lowered the barriers for the public to use AI and harness its power. We introduce PARTICIP-AI, a framework for laypeople to speculate and assess AI use cases and their impacts.
  arXiv  Detail & Related papers  (2024-03-21T19:12:37Z)
• Towards Responsible AI in Banking: Addressing Bias for Fair Decision-Making [69.44075077934914]
  "Responsible AI" emphasizes the critical nature of addressing biases within the development of a corporate culture. This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias. In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages.
  arXiv  Detail & Related papers  (2024-01-13T14:07:09Z)
• AI Deception: A Survey of Examples, Risks, and Potential Solutions [20.84424818447696]
  This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth.
  arXiv  Detail & Related papers  (2023-08-28T17:59:35Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Inherent Limitations of AI Fairness [16.588468396705366]
  The study of AI fairness has rapidly developed into a rich field of research with links to computer science, social science, law, and philosophy. Many technical solutions for measuring and achieving AI fairness have been proposed, yet their approach has been criticized in recent years for being misleading, unrealistic and harmful.
  arXiv  Detail & Related papers  (2022-12-13T11:23:24Z)
• Never trust, always verify : a roadmap for Trustworthy AI? [12.031113181911627]
  We examine trust in the context of AI-based systems to understand what it means for an AI system to be trustworthy. We suggest a trust (resp. zero-trust) model for AI and suggest a set of properties that should be satisfied to ensure the trustworthiness of AI systems.
  arXiv  Detail & Related papers  (2022-06-23T21:13:10Z)
• Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance [14.043062659347427]
  Laws are being proposed and passed by governments around the world to regulate Artificial Intelligence (AI) systems implemented into the public and private sectors. Many of these regulations address the transparency of AI systems, and related citizen-aware issues like allowing individuals to have the right to an explanation about how an AI system makes a decision that impacts them. We propose a novel stakeholder-first approach that assists technologists in designing transparent, regulatory compliant systems.
  arXiv  Detail & Related papers  (2022-06-10T09:39:00Z)
• A Human-Centric Assessment Framework for AI [11.065260433086024]
  There is no agreed standard on how explainable AI systems should be assessed. Inspired by the Turing test, we introduce a human-centric assessment framework. This setup can serve as framework for a wide range of human-centric AI system assessments.
  arXiv  Detail & Related papers  (2022-05-25T12:59:13Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.