Examining risks of racial biases in NLP tools for child protective services

• URL: http://arxiv.org/abs/2305.19409v1
• Date: Tue, 30 May 2023 21:00:47 GMT
• Title: Examining risks of racial biases in NLP tools for child protective services
• Authors: Anjalie Field, Amanda Coston, Nupoor Gandhi, Alexandra Chouldechova, Emily Putnam-Hornstein, David Steier, Yulia Tsvetkov
• Abstract summary: We focus on one such setting: child protective services (CPS) Given well-established racial bias in this setting, we investigate possible ways deployed NLP is liable to increase racial disparities. We document consistent algorithmic unfairness in NER models, possible algorithmic unfairness in coreference resolution models, and little evidence of exacerbated racial bias in risk prediction.
• Score: 78.81107364902958
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Although much literature has established the presence of demographic bias in natural language processing (NLP) models, most work relies on curated bias metrics that may not be reflective of real-world applications. At the same time, practitioners are increasingly using algorithmic tools in high-stakes settings, with particular recent interest in NLP. In this work, we focus on one such setting: child protective services (CPS). CPS workers often write copious free-form text notes about families they are working with, and CPS agencies are actively seeking to deploy NLP models to leverage these data. Given well-established racial bias in this setting, we investigate possible ways deployed NLP is liable to increase racial disparities. We specifically examine word statistics within notes and algorithmic fairness in risk prediction, coreference resolution, and named entity recognition (NER). We document consistent algorithmic unfairness in NER models, possible algorithmic unfairness in coreference resolution models, and little evidence of exacerbated racial bias in risk prediction. While there is existing pronounced criticism of risk prediction, our results expose previously undocumented risks of racial bias in realistic information extraction systems, highlighting potential concerns in deploying them, even though they may appear more benign. Our work serves as a rare realistic examination of NLP algorithmic fairness in a potential deployed setting and a timely investigation of a specific risk associated with deploying NLP in CPS settings.

Related papers

• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
  We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods. We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results. We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
  arXiv  Detail & Related papers  (2024-07-19T14:53:18Z)
• DispaRisk: Auditing Fairness Through Usable Information [21.521208250966918]
  DispaRisk is a framework designed to assess the potential risks of disparities in datasets during the initial stages of machine learning pipeline. DispaRisk identifies datasets with a high risk of discrimination, detect model families prone to biases within an ML pipeline, and enhance the explainability of these bias risks. This work contributes to the development of fairer ML systems by providing a robust tool for early bias detection and mitigation.
  arXiv  Detail & Related papers  (2024-05-20T20:56:01Z)
• Beyond Predictive Algorithms in Child Welfare [12.994514011716138]
  Caseworkers in the child welfare (CW) sector use predictive decision-making algorithms built on risk assessment (RA) data to guide and support CW decisions. Our study finds that common risk metrics used to assess families and build CWS predictive risk models (PRMs) are unable to predict discharge outcomes for children who are not reunified with their birth parent(s) Although casenotes cannot predict discharge outcomes, they contain contextual case signals.
  arXiv  Detail & Related papers  (2024-02-26T08:59:46Z)
• Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articles [10.8637226966191]
  We investigate the potential for nationality biases in natural language processing (NLP) models using human evaluation methods. Our study employs a two-step mixed-methods approach to identify and understand the impact of nationality bias in a text generation model. Our findings reveal that biased NLP models tend to replicate and amplify existing societal biases, which can translate to harm if used in a sociotechnical setting.
  arXiv  Detail & Related papers  (2023-08-08T15:46:27Z)
• Are fairness metric scores enough to assess discrimination biases in machine learning? [4.073786857780967]
  We focus on the Bios dataset, and our learning task is to predict the occupation of individuals, based on their biography. We address an important limitation of theoretical discussions dealing with group-wise fairness metrics: they focus on large datasets. We then question how reliable are different popular measures of bias when the size of the training set is simply sufficient to learn reasonably accurate predictions.
  arXiv  Detail & Related papers  (2023-06-08T15:56:57Z)
• Uncertainty in Natural Language Processing: Sources, Quantification, and Applications [56.130945359053776]
  We provide a comprehensive review of uncertainty-relevant works in the NLP field. We first categorize the sources of uncertainty in natural language into three types, including input, system, and output. We discuss the challenges of uncertainty estimation in NLP and discuss potential future directions.
  arXiv  Detail & Related papers  (2023-06-05T06:46:53Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• A Survey on Bias and Fairness in Natural Language Processing [1.713291434132985]
  We analyze the origins of biases, the definitions of fairness, and how different subfields of NLP bias can be mitigated. We discuss how future studies can work towards eradicating pernicious biases from NLP algorithms.
  arXiv  Detail & Related papers  (2022-03-06T18:12:30Z)
• Statistical discrimination in learning agents [64.78141757063142]
  Statistical discrimination emerges in agent policies as a function of both the bias in the training population and of agent architecture. We show that less discrimination emerges with agents that use recurrent neural networks, and when their training environment has less bias.
  arXiv  Detail & Related papers  (2021-10-21T18:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.