Addressing multiple metrics of group fairness in data-driven decision making

• URL: http://arxiv.org/abs/2003.04794v1
• Date: Tue, 10 Mar 2020 15:13:05 GMT
• Title: Addressing multiple metrics of group fairness in data-driven decision making
• Abstract summary: FAT-ML literature proposes a varied set of group fairness metrics to measure discrimination against socio-demographic groups. We observe that several of these metrics cluster together in two or three main clusters for the same groups and machine learning methods. We propose a robust way to visualize multidimensional fairness in two dimensions through a Principal Component Analysis (PCA) of the group fairness metrics.
• Score: 2.750586731051391
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The Fairness, Accountability, and Transparency in Machine Learning (FAT-ML) literature proposes a varied set of group fairness metrics to measure discrimination against socio-demographic groups that are characterized by a protected feature, such as gender or race.Such a system can be deemed as either fair or unfair depending on the choice of the metric. Several metrics have been proposed, some of them incompatible with each other.We do so empirically, by observing that several of these metrics cluster together in two or three main clusters for the same groups and machine learning methods. In addition, we propose a robust way to visualize multidimensional fairness in two dimensions through a Principal Component Analysis (PCA) of the group fairness metrics. Experimental results on multiple datasets show that the PCA decomposition explains the variance between the metrics with one to three components.

Related papers

• Fairness in multi-class multi-group classification problems via contextial coherent risk measures [0.42970700836450487]
  We propose a new design of fair classifiers for multi-class classification problems in the presence of vector-valued sensitive attributes.<n>We demonstrate the advantages of the proposed framework in comparison to the support-vector machine framework and other methods handling fairness.
  arXiv  Detail & Related papers  (2026-08-31T04:11:21Z)
• MESD: Detecting and Mitigating Procedural Bias in Intersectional Groups [0.0]
  This research addresses an important gap by examining explainability with respect to fairness across multiple protected categories.<n> Experimental results across multiple datasets show that UEF effectively balances objectives.
  arXiv  Detail & Related papers  (2026-03-13T15:42:31Z)
• fairmetrics: An R package for group fairness evaluation [0.40964539027092906]
  fairmetrics R package offers a user-friendly framework for rigorously evaluating group-based fairness criteria.<n>Group-based fairness criteria assess whether a model is equally accurate or well-calibrated across a set of predefined groups.<n>fairmetrics provides both point and interval estimates for multiple metrics through a convenient wrapper function.
  arXiv  Detail & Related papers  (2025-06-06T17:07:41Z)
• Quantifying Query Fairness Under Unawareness [82.33181164973365]
  We introduce a robust fairness estimator based on quantification that effectively handles multiple sensitive attributes beyond binary classifications.<n>Our method outperforms existing baselines across various sensitive attributes and is the first to establish a reliable protocol for measuring fairness under unawareness.
  arXiv  Detail & Related papers  (2025-06-04T16:31:44Z)
• FACEGroup: Feasible and Actionable Counterfactual Explanations for Group Fairness [6.453283750455735]
  This paper introduces the first graph-based framework for generating group counterfactual explanations to audit group fairness.<n>Our framework, FACEGroup, models real-world feasibility constraints, identifies subgroups with similar counterfactuals, and captures key trade-offs in counterfactual generation.
  arXiv  Detail & Related papers  (2024-10-29T23:10:01Z)
• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Fair Abstractive Summarization of Diverse Perspectives [103.08300574459783]
  A fair summary should provide a comprehensive coverage of diverse perspectives without underrepresenting certain groups. We first formally define fairness in abstractive summarization as not underrepresenting perspectives of any groups of people. We propose four reference-free automatic metrics by measuring the differences between target and source perspectives.
  arXiv  Detail & Related papers  (2023-11-14T03:38:55Z)
• A Canonical Data Transformation for Achieving Inter- and Within-group Fairness [17.820200610132265]
  We introduce a formal definition of within-group fairness that maintains fairness among individuals from within the same group. We propose a pre-processing framework to meet both inter- and within-group fairness criteria with little compromise in accuracy. We apply this framework to the COMPAS risk assessment and Law School datasets and compare its performance to two regularization-based methods.
  arXiv  Detail & Related papers  (2023-10-23T17:00:20Z)
• Counterpart Fairness -- Addressing Systematic between-group Differences in Fairness Evaluation [17.495053606192375]
  When using machine learning to aid decision-making, it is critical to ensure that an algorithmic decision is fair and does not discriminate against specific individuals/groups. Existing group fairness methods aim to ensure equal outcomes across groups delineated by protected variables like race or gender. In cases where systematic differences between groups play a significant role in outcomes, these methods may overlook the influence of non-protected variables.
  arXiv  Detail & Related papers  (2023-05-29T15:41:12Z)
• Learning Informative Representation for Fairness-aware Multivariate Time-series Forecasting: A Group-based Perspective [50.093280002375984]
  Performance unfairness among variables widely exists in multivariate time series (MTS) forecasting models. We propose a novel framework, named FairFor, for fairness-aware MTS forecasting.
  arXiv  Detail & Related papers  (2023-01-27T04:54:12Z)
• Estimating Structural Disparities for Face Models [54.062512989859265]
  In machine learning, disparity metrics are often defined by measuring the difference in the performance or outcome of a model, across different sub-populations. We explore performing such analysis on computer vision models trained on human faces, and on tasks such as face attribute prediction and affect estimation.
  arXiv  Detail & Related papers  (2022-04-13T05:30:53Z)
• Measuring Fairness of Text Classifiers via Prediction Sensitivity [63.56554964580627]
  ACCUMULATED PREDICTION SENSITIVITY measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features. We show that the metric can be theoretically linked with a specific notion of group fairness (statistical parity) and individual fairness.
  arXiv  Detail & Related papers  (2022-03-16T15:00:33Z)
• Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics [5.74271110290378]
  We evaluate a set of metrics originating from economics, distributional inequality metrics, and their ability to measure disparities in content exposure in the Twitter algorithmic timeline. We show that we can use these metrics to identify content suggestion algorithms that contribute more strongly to skewed outcomes between users.
  arXiv  Detail & Related papers  (2022-02-03T14:41:39Z)
• Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics [5.248359551210087]
  Measuring bias is key for better understanding and addressing unfairness in NLP/ML models. We shed more light on the differences and similarities between the fairness metrics used in NLP.
  arXiv  Detail & Related papers  (2021-06-28T11:02:33Z)
• MultiFair: Multi-Group Fairness in Machine Learning [52.24956510371455]
  We study multi-group fairness in machine learning (MultiFair) We propose a generic end-to-end algorithmic framework to solve it. Our proposed framework is generalizable to many different settings.
  arXiv  Detail & Related papers  (2021-05-24T02:30:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.