Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

• URL: http://arxiv.org/abs/2112.05675v1
• Date: Fri, 10 Dec 2021 17:14:34 GMT
• Title: Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
• Authors: Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, Hanna Wallach
• Abstract summary: We conduct interviews and workshops with AI practitioners to identify practitioners' processes, challenges, and needs for support. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups. We identify impacts on fairness work stemming from a lack of engagement with direct stakeholders, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.
• Score: 18.148737010217953
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Various tools and practices have been developed to support practitioners in identifying, assessing, and mitigating fairness-related harms caused by AI systems. However, prior research has highlighted gaps between the intended design of these tools and practices and their use within particular contexts, including gaps caused by the role that organizational factors play in shaping fairness work. In this paper, we investigate these gaps for one such practice: disaggregated evaluations of AI systems, intended to uncover performance disparities between demographic groups. By conducting semi-structured interviews and structured workshops with thirty-three AI practitioners from ten teams at three technology companies, we identify practitioners' processes, challenges, and needs for support when designing disaggregated evaluations. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups on which to focus, and collecting datasets with which to conduct disaggregated evaluations. More generally, we identify impacts on fairness work stemming from a lack of engagement with direct stakeholders, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.

Related papers

• Preliminary Insights on Industry Practices for Addressing Fairness Debt [4.546982900370235]
  This study explores how software professionals identify and address biases in AI systems within the software industry. Our paper presents initial evidence on addressing fairness debt and provides a foundation for developing structured guidelines to manage fairness-related issues in AI systems.
  arXiv  Detail & Related papers  (2024-09-04T04:18:42Z)
• Fairness in AI-Driven Recruitment: Challenges, Metrics, Methods, and Future Directions [0.0]
  Big data and machine learning has led to a rapid transformation in the traditional recruitment process. Given the prevalence of AI-based recruitment, there is growing concern that human biases may carry over to decisions made by these systems. This paper provides a comprehensive overview of this emerging field by discussing the types of biases encountered in AI-driven recruitment.
  arXiv  Detail & Related papers  (2024-05-30T05:25:14Z)
• The Impossibility of Fair LLMs [59.424918263776284]
  The need for fair AI is increasingly clear in the era of large language models (LLMs) We review the technical frameworks that machine learning researchers have used to evaluate fairness. We develop guidelines for the more realistic goal of achieving fairness in particular use cases.
  arXiv  Detail & Related papers  (2024-05-28T04:36:15Z)
• Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice [10.979734542685447]
  An emerging body of research indicates that ineffective cross-functional collaboration represents a major barrier to addressing issues of fairness in AI design and development. We conducted a series of interviews and design workshops with 23 industry practitioners spanning various roles from 17 companies. We found that practitioners engaged in bridging work to overcome frictions in understanding, contextualization, and evaluation around AI fairness across roles.
  arXiv  Detail & Related papers  (2023-06-10T23:42:26Z)
• Perspectives on Large Language Models for Relevance Judgment [56.935731584323996]
  Large language models (LLMs) claim that they can assist with relevance judgments. It is not clear whether automated judgments can reliably be used in evaluations of retrieval systems.
  arXiv  Detail & Related papers  (2023-04-13T13:08:38Z)
• Causal Fairness Analysis [68.12191782657437]
  We introduce a framework for understanding, modeling, and possibly solving issues of fairness in decision-making settings. The main insight of our approach will be to link the quantification of the disparities present on the observed data with the underlying, and often unobserved, collection of causal mechanisms. Our effort culminates in the Fairness Map, which is the first systematic attempt to organize and explain the relationship between different criteria found in the literature.
  arXiv  Detail & Related papers  (2022-07-23T01:06:34Z)
• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach [131.20444904674494]
  We tackle the problem of measuring group fairness under unawareness of sensitive attributes. We show that quantification approaches are particularly suited to tackle the fairness-under-unawareness problem.
  arXiv  Detail & Related papers  (2021-09-17T13:45:46Z)
• Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs [42.401239658653914]
  We argue that a deeper understanding of the choices, considerations, and tradeoffs involved in designing disaggregated evaluations will better enable researchers, practitioners, and the public to understand the ways in which AI systems may be underperforming for particular groups of people.
  arXiv  Detail & Related papers  (2021-03-10T14:26:14Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)
• No computation without representation: Avoiding data and algorithm biases through diversity [11.12971845021808]
  We draw connections between the lack of diversity within academic and professional computing fields and the type and breadth of the biases encountered in datasets. We use these lessons to develop recommendations that provide concrete steps for the computing community to increase diversity.
  arXiv  Detail & Related papers  (2020-02-26T23:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.