Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing
- URL: http://arxiv.org/abs/2207.10384v2
- Date: Fri, 16 Jun 2023 10:00:02 GMT
- Title: Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing
- Authors: Alexander Brown, Nenad Tomasev, Jan Freyberg, Yuan Liu, Alan
Karthikesalingam, Jessica Schrouff
- Abstract summary: Machine learning holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities.
One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data.
Using multi-task learning, we propose the first method to assess and mitigate shortcut learning as a part of the fairness assessment of clinical ML systems.
- Score: 62.9062883851246
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning (ML) holds great promise for improving healthcare, but it is
critical to ensure that its use will not propagate or amplify health
disparities. An important step is to characterize the (un)fairness of ML models
- their tendency to perform differently across subgroups of the population -
and to understand its underlying mechanisms. One potential driver of
algorithmic unfairness, shortcut learning, arises when ML models base
predictions on improper correlations in the training data. However, diagnosing
this phenomenon is difficult, especially when sensitive attributes are causally
linked with disease. Using multi-task learning, we propose the first method to
assess and mitigate shortcut learning as a part of the fairness assessment of
clinical ML systems, and demonstrate its application to clinical tasks in
radiology and dermatology. Finally, our approach reveals instances when
shortcutting is not responsible for unfairness, highlighting the need for a
holistic approach to fairness mitigation in medical AI.
Related papers
- Expert Study on Interpretable Machine Learning Models with Missing Data [10.637366819633302]
Inherently interpretable machine learning (IML) models provide valuable insights for clinical decision-making but face challenges when features have missing values.
We conducted a survey with 71 clinicians from 29 trauma centers across France to study the interaction between medical professionals and IML applied to data with missing values.
arXiv Detail & Related papers (2024-11-14T17:02:41Z) - XAI4LLM. Let Machine Learning Models and LLMs Collaborate for Enhanced In-Context Learning in Healthcare [16.79952669254101]
We develop a novel method for zero-shot/few-shot in-context learning (ICL) using a multi-layered structured prompt.
We also explore the efficacy of two communication styles between the user and Large Language Models (LLMs)
Our study systematically evaluates the diagnostic accuracy and risk factors, including gender bias and false negative rates.
arXiv Detail & Related papers (2024-05-10T06:52:44Z) - Interpretable Machine Learning for Survival Analysis [3.618561939712435]
interpretable machine learning (IML) or explainable artificial intelligence (XAI) has become increasingly important over the last decade.
Lack of readily available IML methods may have deterred medical practitioners and policy makers in public health from leveraging the full potential of machine learning.
We present a review of the limited existing amount of work on IML methods for survival analysis within the context of the general IML taxonomy.
arXiv Detail & Related papers (2024-03-15T12:38:00Z) - Federated Learning for Early Dropout Prediction on Healthy Ageing
Applications [0.0]
We present a federated machine learning (FML) approach that minimizes privacy concerns and enables distributed training, without transferring individual data.
Our results show that data selection and class imbalance handling techniques significantly improve the predictive accuracy of models trained under FML.
arXiv Detail & Related papers (2023-09-08T13:17:06Z) - Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning.
They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health.
Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
arXiv Detail & Related papers (2023-05-30T22:05:11Z) - Artificial Intelligence for Dementia Research Methods Optimization [0.49050354212898845]
We present an overview of machine learning algorithms most frequently used in dementia research.
We discuss issues of replicability and interpretability and how these impact the clinical applicability of dementia research.
We give examples of how state-of-the-art methods, such as transfer learning, multi-task learning, and reinforcement learning, may be applied to overcome these issues.
arXiv Detail & Related papers (2023-03-02T08:50:25Z) - Auditing Algorithmic Fairness in Machine Learning for Health with
Severity-Based LOGAN [70.76142503046782]
We propose supplementing machine learning-based (ML) healthcare tools for bias with SLOGAN, an automatic tool for capturing local biases in a clinical prediction task.
LOGAN adapts an existing tool, LOcal Group biAs detectioN, by contextualizing group bias detection in patient illness severity and past medical history.
On average, SLOGAN identifies larger fairness disparities in over 75% of patient groups than LOGAN while maintaining clustering quality.
arXiv Detail & Related papers (2022-11-16T08:04:12Z) - Fair Machine Learning in Healthcare: A Review [90.22219142430146]
We analyze the intersection of fairness in machine learning and healthcare disparities.
We provide a critical review of the associated fairness metrics from a machine learning standpoint.
We propose several new research directions that hold promise for developing ethical and equitable ML applications in healthcare.
arXiv Detail & Related papers (2022-06-29T04:32:10Z) - Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system.
Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model.
We evaluate our framework on a large-scale public-available skin lesion dataset.
arXiv Detail & Related papers (2021-03-07T03:10:32Z) - Inheritance-guided Hierarchical Assignment for Clinical Automatic
Diagnosis [50.15205065710629]
Clinical diagnosis, which aims to assign diagnosis codes for a patient based on the clinical note, plays an essential role in clinical decision-making.
We propose a novel framework to combine the inheritance-guided hierarchical assignment and co-occurrence graph propagation for clinical automatic diagnosis.
arXiv Detail & Related papers (2021-01-27T13:16:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.