Imputation Strategies Under Clinical Presence: Impact on Algorithmic Fairness

• URL: http://arxiv.org/abs/2208.06648v4
• Date: Mon, 17 Mar 2025 23:15:24 GMT
• Title: Imputation Strategies Under Clinical Presence: Impact on Algorithmic Fairness
• Authors: Vincent Jeanselme, Maria De-Arteaga, Zhe Zhang, Jessica Barrett, Brian Tom,
• Abstract summary: We argue that machine learning risks reinforcing biases present in data and in what is absent from data.<n>The way we address missingness in healthcare can have detrimental impacts on algorithmic fairness.<n>We propose a framework for empirically guiding imputation choices, and an accompanying reporting framework.
• Score: 8.958956425857878
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning risks reinforcing biases present in data and, as we argue in this work, in what is absent from data. In healthcare, societal and decision biases shape patterns in missing data, yet the algorithmic fairness implications of group-specific missingness are poorly understood. The way we address missingness in healthcare can have detrimental impacts on downstream algorithmic fairness. Our work questions current recommendations and practices aimed at handling missing data with a focus on their effect on algorithmic fairness, and offers a path forward. Specifically, we consider the theoretical underpinnings of existing recommendations as well as their empirical predictive performance and corresponding algorithmic fairness measured through subgroup performances. Our results show that current practices for handling missingness lack principled foundations, are disconnected from the realities of missingness mechanisms in healthcare, and can be counterproductive. For example, we show that favouring group-specific imputation strategy can be misguided and exacerbate prediction disparities. We then build on our findings to propose a framework for empirically guiding imputation choices, and an accompanying reporting framework. Our work constitutes an important contribution to recent efforts by regulators and practitioners to grapple with the realities of real-world data, and to foster the responsible and transparent deployment of machine learning systems. We demonstrate the practical utility of the proposed framework through experimentation on widely used datasets, where we show how the proposed framework can guide the selection of imputation strategies, allowing us to choose among strategies that yield equal overall predictive performance but present different algorithmic fairness properties.

Related papers

• Targeted Learning for Data Fairness [52.59573714151884]
  We expand fairness inference by evaluating fairness in the data generating process itself. We derive estimators demographic parity, equal opportunity, and conditional mutual information. To validate our approach, we perform several simulations and apply our estimators to real data.
  arXiv  Detail & Related papers  (2025-02-06T18:51:28Z)
• Optimisation Strategies for Ensuring Fairness in Machine Learning: With and Without Demographics [4.662958544712181]
  This paper introduces two formal frameworks to tackle open questions in machine learning fairness. In one framework, operator-valued optimisation and min-max objectives are employed to address unfairness in time-series problems. In the second framework, the challenge of lacking sensitive attributes, such as gender and race, in commonly used datasets is addressed.
  arXiv  Detail & Related papers  (2024-11-13T22:29:23Z)
• DispaRisk: Auditing Fairness Through Usable Information [21.521208250966918]
  DispaRisk is a framework designed to assess the potential risks of disparities in datasets during the initial stages of machine learning pipeline. DispaRisk identifies datasets with a high risk of discrimination, detect model families prone to biases within an ML pipeline, and enhance the explainability of these bias risks. This work contributes to the development of fairer ML systems by providing a robust tool for early bias detection and mitigation.
  arXiv  Detail & Related papers  (2024-05-20T20:56:01Z)
• The Impact of Differential Feature Under-reporting on Algorithmic Fairness [86.275300739926]
  We present an analytically tractable model of differential feature under-reporting. We then use to characterize the impact of this kind of data bias on algorithmic fairness. Our results show that, in real world data settings, under-reporting typically leads to increasing disparities.
  arXiv  Detail & Related papers  (2024-01-16T19:16:22Z)
• On The Impact of Machine Learning Randomness on Group Fairness [11.747264308336012]
  We investigate the impact on group fairness of different sources of randomness in training neural networks. We show that the variance in group fairness measures is rooted in the high volatility of the learning process on under-represented groups. We show how one can control group-level accuracy, with high efficiency and negligible impact on the model's overall performance, by simply changing the data order for a single epoch.
  arXiv  Detail & Related papers  (2023-07-09T09:36:31Z)
• An Operational Perspective to Fairness Interventions: Where and How to Intervene [9.833760837977222]
  We present a holistic framework for evaluating and contextualizing fairness interventions. We demonstrate our framework with a case study on predictive parity. We find predictive parity is difficult to achieve without using group data.
  arXiv  Detail & Related papers  (2023-02-03T07:04:33Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• To Impute or not to Impute? -- Missing Data in Treatment Effect Estimation [84.76186111434818]
  We identify a new missingness mechanism, which we term mixed confounded missingness (MCM), where some missingness determines treatment selection and other missingness is determined by treatment selection. We show that naively imputing all data leads to poor performing treatment effects models, as the act of imputation effectively removes information necessary to provide unbiased estimates. Our solution is selective imputation, where we use insights from MCM to inform precisely which variables should be imputed and which should not.
  arXiv  Detail & Related papers  (2022-02-04T12:08:31Z)
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data [83.50281440043241]
  We study the problem of inferring heterogeneous treatment effects from time-to-event data. We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
  arXiv  Detail & Related papers  (2021-10-26T20:13:17Z)
• TRAPDOOR: Repurposing backdoors to detect dataset bias in machine learning-based genomic analysis [15.483078145498085]
  Under-representation of groups in datasets can lead to inaccurate predictions for certain groups, which can exacerbate systemic discrimination issues. We propose TRAPDOOR, a methodology for identification of biased datasets by repurposing a technique that has been mostly proposed for nefarious purposes: Neural network backdoors. Using a real-world cancer dataset, we analyze the dataset with the bias that already existed towards white individuals and also introduced biases in datasets artificially.
  arXiv  Detail & Related papers  (2021-08-14T17:02:02Z)
• Fair Representation Learning using Interpolation Enabled Disentanglement [9.043741281011304]
  We propose a novel method to address two key issues: (a) Can we simultaneously learn fair disentangled representations while ensuring the utility of the learned representation for downstream tasks, and (b)Can we provide theoretical insights into when the proposed approach will be both fair and accurate. To address the former, we propose the method FRIED, Fair Representation learning using Interpolation Enabled Disentanglement.
  arXiv  Detail & Related papers  (2021-07-31T17:32:12Z)
• Doing Great at Estimating CATE? On the Neglected Assumptions in Benchmark Comparisons of Treatment Effect Estimators [91.3755431537592]
  We show that even in arguably the simplest setting, estimation under ignorability assumptions can be misleading. We consider two popular machine learning benchmark datasets for evaluation of heterogeneous treatment effect estimators. We highlight that the inherent characteristics of the benchmark datasets favor some algorithms over others.
  arXiv  Detail & Related papers  (2021-07-28T13:21:27Z)
• Fair Classification with Group-Dependent Label Noise [6.324366770332667]
  This work examines how to train fair classifiers in settings where training labels are corrupted with random noise. We show that naively imposing parity constraints on demographic disparity measures, without accounting for heterogeneous and group-dependent error rates, can decrease both the accuracy and the fairness of the resulting classifier.
  arXiv  Detail & Related papers  (2020-10-31T22:35:01Z)
• Targeted VAE: Variational and Targeted Learning for Causal Inference [39.351088248776435]
  Undertaking causal inference with observational data is incredibly useful across a wide range of tasks. There are two significant challenges associated with undertaking causal inference using observational data. We address these two challenges by combining structured inference and targeted learning.
  arXiv  Detail & Related papers  (2020-09-28T16:55:24Z)
• Beyond Individual and Group Fairness [90.4666341812857]
  We present a new data-driven model of fairness that is guided by the unfairness complaints received by the system. Our model supports multiple fairness criteria and takes into account their potential incompatibilities.
  arXiv  Detail & Related papers  (2020-08-21T14:14:44Z)
• An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction [7.945729033499554]
  The use of machine learning to guide clinical decision making has the potential to worsen existing health disparities. Several recent works frame the problem as that of algorithmic fairness, a framework that has attracted considerable attention and criticism. We conduct an empirical study to characterize the impact of penalizing group fairness violations on an array of measures of model performance and group fairness.
  arXiv  Detail & Related papers  (2020-07-20T17:46:31Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects [61.03579766573421]
  We study estimation of individual-level causal effects, such as a single patient's response to alternative medication. We devise representation learning algorithms that minimize our bound, by regularizing the representation's induced treatment group distance. We extend these algorithms to simultaneously learn a weighted representation to further reduce treatment group distances.
  arXiv  Detail & Related papers  (2020-01-21T10:16:33Z)
• Learning Overlapping Representations for the Estimation of Individualized Treatment Effects [97.42686600929211]
  Estimating the likely outcome of alternatives from observational data is a challenging problem. We show that algorithms that learn domain-invariant representations of inputs are often inappropriate. We develop a deep kernel regression algorithm and posterior regularization framework that substantially outperforms the state-of-the-art on a variety of benchmarks data sets.
  arXiv  Detail & Related papers  (2020-01-14T12:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.