Assessing Social Determinants-Related Performance Bias of Machine Learning Models: A case of Hyperchloremia Prediction in ICU Population

• URL: http://arxiv.org/abs/2111.09507v1
• Date: Thu, 18 Nov 2021 03:58:50 GMT
• Title: Assessing Social Determinants-Related Performance Bias of Machine Learning Models: A case of Hyperchloremia Prediction in ICU Population
• Authors: Songzi Liu, Yuan Luo
• Abstract summary: We evaluated four classifiers built to predict Hyperchloremia, a condition that often results from aggressive fluids administration in the ICU population. We observed that adding social determinants features in addition to the lab-based ones improved model performance on all patients. We urge future researchers to design models that proactively adjust for potential biases and include subgroup reporting.
• Score: 6.8473641147443995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning in medicine leverages the wealth of healthcare data to extract knowledge, facilitate clinical decision-making, and ultimately improve care delivery. However, ML models trained on datasets that lack demographic diversity could yield suboptimal performance when applied to the underrepresented populations (e.g. ethnic minorities, lower social-economic status), thus perpetuating health disparity. In this study, we evaluated four classifiers built to predict Hyperchloremia - a condition that often results from aggressive fluids administration in the ICU population - and compared their performance in racial, gender, and insurance subgroups. We observed that adding social determinants features in addition to the lab-based ones improved model performance on all patients. The subgroup testing yielded significantly different AUC scores in 40 out of the 44 model-subgroup, suggesting disparities when applying ML models to social determinants subgroups. We urge future researchers to design models that proactively adjust for potential biases and include subgroup reporting in their studies.

Related papers

• Using Backbone Foundation Model for Evaluating Fairness in Chest Radiography Without Demographic Data [2.7436483977171333]
  This study aims to investigate the effectiveness of using the backbone of Foundation Models as an embedding extractor. We propose utilizing these groups in different stages of bias mitigation, including pre-processing, in-processing, and evaluation.
  arXiv  Detail & Related papers  (2024-08-28T20:35:38Z)
• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• Evaluating the Fairness of the MIMIC-IV Dataset and a Baseline Algorithm: Application to the ICU Length of Stay Prediction [65.268245109828]
  This paper uses the MIMIC-IV dataset to examine the fairness and bias in an XGBoost binary classification model predicting the ICU length of stay. The research reveals class imbalances in the dataset across demographic attributes and employs data preprocessing and feature extraction. The paper concludes with recommendations for fairness-aware machine learning techniques for mitigating biases and the need for collaborative efforts among healthcare professionals and data scientists.
  arXiv  Detail & Related papers  (2023-12-31T16:01:48Z)
• Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neuroscience [21.420302408947194]
  We develop a weighted empirical risk minimization approach that optimally combines data from a source group to make predictions on a target group. We apply this method to multi-source data of 15,363 individuals from 20 neuroimaging studies to build ML models for diagnosis of Alzheimer's disease and estimation of brain age.
  arXiv  Detail & Related papers  (2023-08-06T18:05:39Z)
• Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes [72.13373216644021]
  We study the societal impact of machine learning by considering the collection of models that are deployed in a given context. We find deployed machine learning is prone to systemic failure, meaning some users are exclusively misclassified by all models available. These examples demonstrate ecosystem-level analysis has unique strengths for characterizing the societal impact of machine learning.
  arXiv  Detail & Related papers  (2023-07-12T01:11:52Z)
• Fairness in Machine Learning meets with Equity in Healthcare [6.842248432925292]
  This study proposes an artificial intelligence framework for identifying and mitigating biases in data and models. A case study is presented to demonstrate how systematic biases in data can lead to amplified biases in model predictions. Future research aims to test and validate the proposed ML framework in real-world clinical settings to evaluate its impact on promoting health equity.
  arXiv  Detail & Related papers  (2023-05-11T14:25:34Z)
• Auditing ICU Readmission Rates in an Clinical Database: An Analysis of Risk Factors and Clinical Outcomes [0.0]
  This study presents a machine learning pipeline for clinical data classification in the context of a 30-day readmission problem. The fairness audit uncovers disparities in equal opportunity, predictive parity, false positive rate parity, and false negative rate parity criteria. The study suggests the need for collaborative efforts among researchers, policymakers, and practitioners to address bias and fairness in artificial intelligence (AI) systems.
  arXiv  Detail & Related papers  (2023-04-12T17:09:38Z)
• Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN [70.76142503046782]
  We propose supplementing machine learning-based (ML) healthcare tools for bias with SLOGAN, an automatic tool for capturing local biases in a clinical prediction task. LOGAN adapts an existing tool, LOcal Group biAs detectioN, by contextualizing group bias detection in patient illness severity and past medical history. On average, SLOGAN identifies larger fairness disparities in over 75% of patient groups than LOGAN while maintaining clustering quality.
  arXiv  Detail & Related papers  (2022-11-16T08:04:12Z)
• Avoiding Biased Clinical Machine Learning Model Performance Estimates in the Presence of Label Selection [3.3944964838781093]
  We describe three classes of label selection and simulate five causally distinct scenarios to assess how particular selection mechanisms bias a suite of commonly reported binary machine learning model performance metrics. We find that naive estimates of AUROC on the observed population undershoot actual performance by up to 20%. Such a disparity could be large enough to lead to the wrongful termination of a successful clinical decision support tool.
  arXiv  Detail & Related papers  (2022-09-15T22:30:14Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• Predictive Modeling of ICU Healthcare-Associated Infections from Imbalanced Data. Using Ensembles and a Clustering-Based Undersampling Approach [55.41644538483948]
  This work is focused on both the identification of risk factors and the prediction of healthcare-associated infections in intensive-care units. The aim is to support decision making addressed at reducing the incidence rate of infections.
  arXiv  Detail & Related papers  (2020-05-07T16:13:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.