Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare

• URL: http://arxiv.org/abs/2202.01906v1
• Date: Thu, 3 Feb 2022 23:23:05 GMT
• Title: Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare
• Authors: Stephen R. Pfohl, Yizhe Xu, Agata Foryciarz, Nikolaos Ignatiadis, Julian Genkins, Nigam H. Shah
• Abstract summary: We evaluate the interplay between measures of model performance, fairness, and the expected utility of decision-making. We conduct an empirical case-study via development of models to estimate the ten-year risk of atherosclerotic cardiovascular disease.
• Score: 7.500205995624894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A growing body of work uses the paradigm of algorithmic fairness to frame the development of techniques to anticipate and proactively mitigate the introduction or exacerbation of health inequities that may follow from the use of model-guided decision-making. We evaluate the interplay between measures of model performance, fairness, and the expected utility of decision-making to offer practical recommendations for the operationalization of algorithmic fairness principles for the development and evaluation of predictive models in healthcare. We conduct an empirical case-study via development of models to estimate the ten-year risk of atherosclerotic cardiovascular disease to inform statin initiation in accordance with clinical practice guidelines. We demonstrate that approaches that incorporate fairness considerations into the model training objective typically do not improve model performance or confer greater net benefit for any of the studied patient populations compared to the use of standard learning paradigms followed by threshold selection concordant with patient preferences, evidence of intervention effectiveness, and model calibration. These results hold when the measured outcomes are not subject to differential measurement error across patient populations and threshold selection is unconstrained, regardless of whether differences in model performance metrics, such as in true and false positive error rates, are present. In closing, we argue for focusing model development efforts on developing calibrated models that predict outcomes well for all patient populations while emphasizing that such efforts are complementary to transparent reporting, participatory design, and reasoning about the impact of model-informed interventions in context.

Related papers

• Addressing Data Heterogeneity in Federated Learning of Cox Proportional Hazards Models [8.798959872821962]
  This paper outlines an approach in the domain of federated survival analysis, specifically the Cox Proportional Hazards (CoxPH) model. We present an FL approach that employs feature-based clustering to enhance model accuracy across synthetic datasets and real-world applications.
  arXiv  Detail & Related papers  (2024-07-20T18:34:20Z)
• Ethical considerations of use of hold-out sets in clinical prediction model management [0.4194295877935868]
  We focus on the ethical principles of beneficence, non-maleficence, autonomy and justice. We also discuss statistical issues arising from different hold-out set sampling methods.
  arXiv  Detail & Related papers  (2024-06-05T11:42:46Z)
• Explainable AI for Fair Sepsis Mortality Predictive Model [3.556697333718976]
  We propose a method that learns a performance-optimized predictive model and employs the transfer learning process to produce a model with better fairness. Our method not only aids in identifying and mitigating biases within the predictive model but also fosters trust among healthcare stakeholders.
  arXiv  Detail & Related papers  (2024-04-19T18:56:46Z)
• Causal Inference via Nonlinear Variable Decorrelation for Healthcare Applications [60.26261850082012]
  We introduce a novel method with a variable decorrelation regularizer to handle both linear and nonlinear confounding. We employ association rules as new representations using association rule mining based on the original features to increase model interpretability.
  arXiv  Detail & Related papers  (2022-09-29T17:44:14Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• What Do You See in this Patient? Behavioral Testing of Clinical NLP Models [69.09570726777817]
  We introduce an extendable testing framework that evaluates the behavior of clinical outcome models regarding changes of the input. We show that model behavior varies drastically even when fine-tuned on the same data and that allegedly best-performing models have not always learned the most medically plausible patterns.
  arXiv  Detail & Related papers  (2021-11-30T15:52:04Z)
• A comparison of approaches to improve worst-case predictive model performance over patient subpopulations [14.175321968797252]
  Predictive models for clinical outcomes that are accurate on average in a patient population may underperform drastically for some subpopulations. We identify approaches for model development and selection that consistently improve disaggregated and worst-case performance over subpopulations. We find that, with relatively few exceptions, no approach performs better, for each patient subpopulation examined, than standard learning procedures.
  arXiv  Detail & Related papers  (2021-08-27T13:10:00Z)
• Learning to Predict with Supporting Evidence: Applications to Clinical Risk Prediction [9.199022926064009]
  The impact of machine learning models on healthcare will depend on the degree of trust that healthcare professionals place in the predictions made by these models. We present a method to provide people with clinical expertise with domain-relevant evidence about why a prediction should be trusted.
  arXiv  Detail & Related papers  (2021-03-04T00:26:32Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• On the model-based stochastic value gradient for continuous reinforcement learning [50.085645237597056]
  We show that simple model-based agents can outperform state-of-the-art model-free agents in terms of both sample-efficiency and final reward. Our findings suggest that model-based policy evaluation deserves closer attention.
  arXiv  Detail & Related papers  (2020-08-28T17:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.