Counterfactual Fairness by Combining Factual and Counterfactual Predictions

• URL: http://arxiv.org/abs/2409.01977v1
• Date: Tue, 3 Sep 2024 15:21:10 GMT
• Title: Counterfactual Fairness by Combining Factual and Counterfactual Predictions
• Authors: Zeyu Zhou, Tianci Liu, Ruqi Bai, Jing Gao, Murat Kocaoglu, David I. Inouye,
• Abstract summary: In high-stake domains such as healthcare and hiring, the role of machine learning (ML) in decision-making raises significant fairness concerns. This work focuses on Counterfactual Fairness (CF), which posits that an ML model's outcome on any individual should remain unchanged if they had belonged to a different demographic group. We provide a theoretical study on the inherent trade-off between CF and predictive performance in a model-agnostic manner.
• Score: 18.950415688199993
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In high-stake domains such as healthcare and hiring, the role of machine learning (ML) in decision-making raises significant fairness concerns. This work focuses on Counterfactual Fairness (CF), which posits that an ML model's outcome on any individual should remain unchanged if they had belonged to a different demographic group. Previous works have proposed methods that guarantee CF. Notwithstanding, their effects on the model's predictive performance remains largely unclear. To fill in this gap, we provide a theoretical study on the inherent trade-off between CF and predictive performance in a model-agnostic manner. We first propose a simple but effective method to cast an optimal but potentially unfair predictor into a fair one without losing the optimality. By analyzing its excess risk in order to achieve CF, we quantify this inherent trade-off. Further analysis on our method's performance with access to only incomplete causal knowledge is also conducted. Built upon it, we propose a performant algorithm that can be applied in such scenarios. Experiments on both synthetic and semi-synthetic datasets demonstrate the validity of our analysis and methods.

Related papers

• Boosted Control Functions [10.503777692702952]
  This work aims to bridge the gap between causal effect estimation and prediction tasks. We establish a novel connection between the field of distribution from machine learning, and simultaneous equation models and control function from econometrics. Within this framework, we propose a strong notion of invariance for a predictive model and compare it with existing (weaker) versions.
  arXiv  Detail & Related papers  (2023-10-09T15:43:46Z)
• Causality-Aided Trade-off Analysis for Machine Learning Fairness [11.149507394656709]
  This paper uses causality analysis as a principled method for analyzing trade-offs between fairness parameters and other crucial metrics in machine learning pipelines. We propose a set of domain-specific optimizations to facilitate accurate causal discovery and a unified, novel interface for trade-off analysis based on well-established causal inference methods.
  arXiv  Detail & Related papers  (2023-05-22T14:14:43Z)
• An Operational Perspective to Fairness Interventions: Where and How to Intervene [9.833760837977222]
  We present a holistic framework for evaluating and contextualizing fairness interventions. We demonstrate our framework with a case study on predictive parity. We find predictive parity is difficult to achieve without using group data.
  arXiv  Detail & Related papers  (2023-02-03T07:04:33Z)
• Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints [51.12047280149546]
  A direct approach for obtaining a fair predictive model is to train the model through optimizing its prediction performance subject to fairness constraints. We formulate the training problem of a fairness-aware machine learning model as an AUC optimization problem subject to a class of AUC-based fairness constraints. We demonstrate the effectiveness of our approach on real-world data under different fairness metrics.
  arXiv  Detail & Related papers  (2022-12-23T22:29:08Z)
• Normalise for Fairness: A Simple Normalisation Technique for Fairness in Regression Machine Learning Problems [46.93320580613236]
  We present a simple, yet effective method based on normalisation (FaiReg) for regression problems. We compare it with two standard methods for fairness, namely data balancing and adversarial training. The results show the superior performance of diminishing the effects of unfairness better than data balancing.
  arXiv  Detail & Related papers  (2022-02-02T12:26:25Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
  We propose a novel scalable method to learn double-robust representations for counterfactual predictions. We make robust and efficient counterfactual predictions for both individual and average treatment effects. The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
  arXiv  Detail & Related papers  (2020-10-15T16:39:26Z)
• Robust Validation: Confident Predictions Even When Distributions Shift [19.327409270934474]
  We describe procedures for robust predictive inference, where a model provides uncertainty estimates on its predictions rather than point predictions. We present a method that produces prediction sets (almost exactly) giving the right coverage level for any test distribution in an $f$-divergence ball around the training population. An essential component of our methodology is to estimate the amount of expected future data shift and build robustness to it.
  arXiv  Detail & Related papers  (2020-08-10T17:09:16Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• FedDANE: A Federated Newton-Type Method [49.9423212899788]
  Federated learning aims to jointly learn low statistical models over massively distributed datasets. We propose FedDANE, an optimization that we adapt from DANE, to handle federated learning.
  arXiv  Detail & Related papers  (2020-01-07T07:44:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.