Algorithmic encoding of protected characteristics and its implications on disparities across subgroups

• URL: http://arxiv.org/abs/2110.14755v1
• Date: Wed, 27 Oct 2021 20:30:57 GMT
• Title: Algorithmic encoding of protected characteristics and its implications on disparities across subgroups
• Authors: Ben Glocker and Stefan Winzeck
• Abstract summary: Machine learning models may pick up undesirable correlations between a patient's racial identity and clinical outcome. Very little is known about how these biases are encoded and how one may reduce or even remove disparate performance.
• Score: 17.415882865534638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It has been rightfully emphasized that the use of AI for clinical decision making could amplify health disparities. A machine learning model may pick up undesirable correlations, for example, between a patient's racial identity and clinical outcome. Such correlations are often present in (historical) data used for model development. There has been an increase in studies reporting biases in disease detection models across patient subgroups. Besides the scarcity of data from underserved populations, very little is known about how these biases are encoded and how one may reduce or even remove disparate performance. There is some speculation whether algorithms may recognize patient characteristics such as biological sex or racial identity, and then directly or indirectly use this information when making predictions. But it remains unclear how we can establish whether such information is actually used. This article aims to shed some light on these issues by exploring new methodology allowing intuitive inspections of the inner working of machine learning models for image-based detection of disease. We also evaluate an effective yet debatable technique for addressing disparities leveraging the automatic prediction of patient characteristics, resulting in models with comparable true and false positive rates across subgroups. Our findings may stimulate the discussion about safe and ethical use of AI.

Related papers

• Towards a Transportable Causal Network Model Based on Observational Healthcare Data [1.333879175460266]
  We propose a novel approach that combines selection diagrams, missingness graphs, causal discovery and prior knowledge into a single graphical model. We learn this model from data comprising two different cohorts of patients. The resulting causal network model is validated by expert clinicians in terms of risk assessment, accuracy and explainability.
  arXiv  Detail & Related papers  (2023-11-13T13:23:31Z)
• Learning and DiSentangling Patient Static Information from Time-series Electronic HEalth Record (STEER) [3.079694232219292]
  Recent work in machine learning for healthcare has raised concerns about patient privacy and algorithmic fairness. Here we systematically investigated the ability of time-series electronic health record data to predict patient static information. We found that not only the raw time-series data, but also learned representations from machine learning models, can be trained to predict a variety of static information.
  arXiv  Detail & Related papers  (2023-09-20T14:54:48Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Deep Learning Discovery of Demographic Biomarkers in Echocardiography [0.3957768262206625]
  We test whether it is possible to predict age, race, and sex from cardiac ultrasound images using deep learning algorithms. We trained video-based convolutional neural networks to predict age, sex, and race. We found that deep learning models were able to identify age and sex, while unable to reliably predict race.
  arXiv  Detail & Related papers  (2022-07-13T16:48:49Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• What Do You See in this Patient? Behavioral Testing of Clinical NLP Models [69.09570726777817]
  We introduce an extendable testing framework that evaluates the behavior of clinical outcome models regarding changes of the input. We show that model behavior varies drastically even when fine-tuned on the same data and that allegedly best-performing models have not always learned the most medically plausible patterns.
  arXiv  Detail & Related papers  (2021-11-30T15:52:04Z)
• Explaining medical AI performance disparities across sites with confounder Shapley value analysis [8.785345834486057]
  Multi-site evaluations are key to diagnosing such disparities. Our framework provides a method for quantifying the marginal and cumulative effect of each type of bias on the overall performance difference. We demonstrate its usefulness in a case study of a deep learning model trained to detect the presence of pneumothorax.
  arXiv  Detail & Related papers  (2021-11-12T18:54:10Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Analysing Risk of Coronary Heart Disease through Discriminative Neural Networks [18.124078832445967]
  In critical applications like diagnostics, this class imbalance cannot be overlooked. We depict how we can handle this class imbalance through neural networks using a discriminative model and contrastive loss.
  arXiv  Detail & Related papers  (2020-06-17T06:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.