Towards Effective Discrimination Testing for Generative AI

• URL: http://arxiv.org/abs/2412.21052v1
• Date: Mon, 30 Dec 2024 16:09:33 GMT
• Title: Towards Effective Discrimination Testing for Generative AI
• Authors: Thomas P. Zollo, Nikita Rajaneesh, Richard Zemel, Talia B. Gillis, Emily Black,
• Abstract summary: Generative AI (GenAI) models present new challenges in regulating against discriminatory behavior. We argue that GenAI fairness research still has not met these challenges; instead, a significant gap remains between existing bias assessment methods and regulatory goals.
• Score: 5.2817059203636845
• License:
• Abstract: Generative AI (GenAI) models present new challenges in regulating against discriminatory behavior. In this paper, we argue that GenAI fairness research still has not met these challenges; instead, a significant gap remains between existing bias assessment methods and regulatory goals. This leads to ineffective regulation that can allow deployment of reportedly fair, yet actually discriminatory, GenAI systems. Towards remedying this problem, we connect the legal and technical literature around GenAI bias evaluation and identify areas of misalignment. Through four case studies, we demonstrate how this misalignment between fairness testing techniques and regulatory goals can result in discriminatory outcomes in real-world deployments, especially in adaptive or complex environments. We offer practical recommendations for improving discrimination testing to better align with regulatory goals and enhance the reliability of fairness assessments in future deployments.

Related papers

• An evidence-based methodology for human rights impact assessment (HRIA) in the development of AI data-intensive systems [49.1574468325115]
  We show that human rights already underpin the decisions in the field of data use. This work presents a methodology and a model for a Human Rights Impact Assessment (HRIA) The proposed methodology is tested in concrete case-studies to prove its feasibility and effectiveness.
  arXiv  Detail & Related papers  (2024-07-30T16:27:52Z)
• Formalising Anti-Discrimination Law in Automated Decision Systems [1.560976479364936]
  We introduce a novel decision-theoretic framework grounded in anti-discrimination law of the United Kingdom. We propose the 'conditional estimation parity' metric, which accounts for estimation error and the underlying data-generating process. Our approach bridges the divide between machine learning fairness metrics and anti-discrimination law, offering a legally grounded framework for developing non-discriminatory automated decision systems.
  arXiv  Detail & Related papers  (2024-06-29T10:59:21Z)
• Generative Discrimination: What Happens When Generative AI Exhibits Bias, and What Can Be Done About It [2.2913283036871865]
  chapter explores how genAI intersects with non-discrimination laws. It highlights two main types of discriminatory outputs: (i) demeaning and abusive content and (ii) subtler biases due to inadequate representation of protected groups. It argues for holding genAI providers and deployers liable for discriminatory outputs and highlights the inadequacy of traditional legal frameworks to address genAI-specific issues.
  arXiv  Detail & Related papers  (2024-06-26T13:32:58Z)
• A Nested Model for AI Design and Validation [0.5120567378386615]
  Despite the need for new regulations, there is a mismatch between regulatory science and AI. A five-layer nested model for AI design and validation aims to address these issues.
  arXiv  Detail & Related papers  (2024-06-08T12:46:12Z)
• Compatibility of Fairness Metrics with EU Non-Discrimination Laws: Demographic Parity & Conditional Demographic Disparity [3.5607241839298878]
  Empirical evidence suggests that algorithmic decisions driven by Machine Learning (ML) techniques threaten to discriminate against legally protected groups or create new sources of unfairness. This work aims at assessing up to what point we can assure legal fairness through fairness metrics and under fairness constraints. Our experiments and analysis suggest that AI-assisted decision-making can be fair from a legal perspective depending on the case at hand and the legal justification.
  arXiv  Detail & Related papers  (2023-06-14T09:38:05Z)
• Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation [61.77881142275982]
  This interdisciplinary position paper considers various concerns surrounding fairness and discrimination in AI, and discusses how AI regulations address them. We first look at AI and fairness through the lenses of law, (AI) industry, sociotechnology, and (moral) philosophy, and present various perspectives. We identify and propose the roles AI Regulation should take to make the endeavor of the AI Act a success in terms of AI fairness concerns.
  arXiv  Detail & Related papers  (2022-06-08T12:32:08Z)
• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach [131.20444904674494]
  We tackle the problem of measuring group fairness under unawareness of sensitive attributes. We show that quantification approaches are particularly suited to tackle the fairness-under-unawareness problem.
  arXiv  Detail & Related papers  (2021-09-17T13:45:46Z)
• Training GANs with Stronger Augmentations via Contrastive Discriminator [80.8216679195]
  We introduce a contrastive representation learning scheme into the GAN discriminator, coined ContraD. This "fusion" enables the discriminators to work with much stronger augmentations without increasing their training instability. Our experimental results show that GANs with ContraD consistently improve FID and IS compared to other recent techniques incorporating data augmentations.
  arXiv  Detail & Related papers  (2021-03-17T16:04:54Z)
• Towards Fair Knowledge Transfer for Imbalanced Domain Adaptation [61.317911756566126]
  We propose a Towards Fair Knowledge Transfer framework to handle the fairness challenge in imbalanced cross-domain learning. Specifically, a novel cross-domain mixup generation is exploited to augment the minority source set with target information to enhance fairness. Our model significantly improves over 20% on two benchmarks in terms of the overall accuracy.
  arXiv  Detail & Related papers  (2020-10-23T06:29:09Z)
• Getting Fairness Right: Towards a Toolbox for Practitioners [2.4364387374267427]
  The potential risk of AI systems unintentionally embedding and reproducing bias has attracted the attention of machine learning practitioners and society at large. This paper proposes to draft a toolbox which helps practitioners to ensure fair AI practices.
  arXiv  Detail & Related papers  (2020-03-15T20:53:50Z)
• Discrimination of POVMs with rank-one effects [62.997667081978825]
  This work provides an insight into the problem of discrimination of positive operator valued measures with rank-one effects. We compare two possible discrimination schemes: the parallel and adaptive ones. We provide an explicit algorithm which allows us to find this adaptive scheme.
  arXiv  Detail & Related papers  (2020-02-13T11:34:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.