Disparate Model Performance and Stability in Machine Learning Clinical Support for Diabetes and Heart Diseases

• URL: http://arxiv.org/abs/2412.19495v1
• Date: Fri, 27 Dec 2024 07:31:14 GMT
• Title: Disparate Model Performance and Stability in Machine Learning Clinical Support for Diabetes and Heart Diseases
• Authors: Ioannis Bilionis, Ricardo C. Berrios, Luis Fernandez-Luque, Carlos Castillo,
• Abstract summary: Investigation reveals widespread sex- and age-related inequities in chronic disease datasets and their derived Machine Learning models. The analysis of data from over 25,000 individuals with chronic diseases revealed mild sex-related disparities, favoring predictive accuracy for males, and significant age-related differences, with better accuracy for younger patients. Notably, older patients showed inconsistent predictive accuracy across seven datasets, linked to higher data complexity and lower model performance.
• Score: 2.9247093478805324
• License:
• Abstract: Machine Learning (ML) algorithms are vital for supporting clinical decision-making in biomedical informatics. However, their predictive performance can vary across demographic groups, often due to the underrepresentation of historically marginalized populations in training datasets. The investigation reveals widespread sex- and age-related inequities in chronic disease datasets and their derived ML models. Thus, a novel analytical framework is introduced, combining systematic arbitrariness with traditional metrics like accuracy and data complexity. The analysis of data from over 25,000 individuals with chronic diseases revealed mild sex-related disparities, favoring predictive accuracy for males, and significant age-related differences, with better accuracy for younger patients. Notably, older patients showed inconsistent predictive accuracy across seven datasets, linked to higher data complexity and lower model performance. This highlights that representativeness in training data alone does not guarantee equitable outcomes, and model arbitrariness must be addressed before deploying models in clinical settings.

Related papers

• Examining Imbalance Effects on Performance and Demographic Fairness of Clinical Language Models [4.390908825243365]
  This study statistically probes the relationship between data imbalance and model performance in ICD code prediction. We analyze imbalances in a standard benchmark data across gender, age, ethnicity, and social determinants of health by state-of-the-art biomedical language models. Our study shows that data imbalance significantly impacts model performance and fairness, but feature similarity to the majority class may be a more critical factor.
  arXiv  Detail & Related papers  (2024-12-23T18:58:11Z)
• Enhancing End Stage Renal Disease Outcome Prediction: A Multi-Sourced Data-Driven Approach [7.212939068975618]
  We utilized data about 10,326 CKD patients, combining their clinical and claims information from 2009 to 2018. A 24-month observation window was identified as optimal for balancing early detection and prediction accuracy. The 2021 eGFR equation improved prediction accuracy and reduced racial bias, notably for African American patients.
  arXiv  Detail & Related papers  (2024-10-02T03:21:01Z)
• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• Learning and DiSentangling Patient Static Information from Time-series Electronic HEalth Record (STEER) [3.079694232219292]
  Recent work in machine learning for healthcare has raised concerns about patient privacy and algorithmic fairness. Here we systematically investigated the ability of time-series electronic health record data to predict patient static information. We found that not only the raw time-series data, but also learned representations from machine learning models, can be trained to predict a variety of static information.
  arXiv  Detail & Related papers  (2023-09-20T14:54:48Z)
• Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neuroscience [21.420302408947194]
  We develop a weighted empirical risk minimization approach that optimally combines data from a source group to make predictions on a target group. We apply this method to multi-source data of 15,363 individuals from 20 neuroimaging studies to build ML models for diagnosis of Alzheimer's disease and estimation of brain age.
  arXiv  Detail & Related papers  (2023-08-06T18:05:39Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Longitudinal modeling of MS patient trajectories improves predictions of disability progression [2.117653457384462]
  This work addresses the task of optimally extracting information from longitudinal patient data in the real-world setting. We show that with machine learning methods suited for patient trajectories modeling, we can predict disability progression of patients in a two-year horizon. Compared to the models available in the literature, this work uses the most complete patient history for MS disease progression prediction.
  arXiv  Detail & Related papers  (2020-11-09T20:48:00Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Temporal Phenotyping using Deep Predictive Clustering of Disease Progression [97.88605060346455]
  We develop a deep learning approach for clustering time-series data, where each cluster comprises patients who share similar future outcomes of interest. Experiments on two real-world datasets show that our model achieves superior clustering performance over state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2020-06-15T20:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.