Fairness in machine learning: against false positive rate equality as a measure of fairness

• URL: http://arxiv.org/abs/2007.02890v1
• Date: Mon, 6 Jul 2020 17:03:58 GMT
• Title: Fairness in machine learning: against false positive rate equality as a measure of fairness
• Authors: Robert Long
• Abstract summary: Two popular fairness measures are calibration and equality of false positive rate. I give an ethical framework for thinking about these measures and argue that false positive rate equality does not track anything about fairness.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As machine learning informs increasingly consequential decisions, different metrics have been proposed for measuring algorithmic bias or unfairness. Two popular fairness measures are calibration and equality of false positive rate. Each measure seems intuitively important, but notably, it is usually impossible to satisfy both measures. For this reason, a large literature in machine learning speaks of a fairness tradeoff between these two measures. This framing assumes that both measures are, in fact, capturing something important. To date, philosophers have not examined this crucial assumption, and examined to what extent each measure actually tracks a normatively important property. This makes this inevitable statistical conflict, between calibration and false positive rate equality, an important topic for ethics. In this paper, I give an ethical framework for thinking about these measures and argue that, contrary to initial appearances, false positive rate equality does not track anything about fairness, and thus sets an incoherent standard for evaluating the fairness of algorithms.

Related papers

• The Unfairness of $\varepsilon$-Fairness [0.0]
  We show that if the concept of $varepsilon$-fairness is employed, it can possibly lead to outcomes that are maximally unfair in the real-world context. We illustrate our findings with two real-world examples: college admissions and credit risk assessment.
  arXiv  Detail & Related papers  (2024-05-15T14:13:35Z)
• Navigating Fairness Measures and Trade-Offs [0.0]
  I show that by using Rawls' notion of justice as fairness, we can create a basis for navigating fairness measures and the accuracy trade-off. This also helps to close part of the gap between philosophical accounts of distributive justice and the fairness literature.
  arXiv  Detail & Related papers  (2023-07-17T13:45:47Z)
• Reconciling Predictive and Statistical Parity: A Causal Approach [68.59381759875734]
  We propose a new causal decomposition formula for the fairness measures associated with predictive parity. We show that the notions of statistical and predictive parity are not really mutually exclusive, but complementary and spanning a spectrum of fairness notions.
  arXiv  Detail & Related papers  (2023-06-08T09:23:22Z)
• Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP [64.45845091719002]
  Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. This paper seeks to clarify the current situation and plot a course for meaningful progress in fair learning.
  arXiv  Detail & Related papers  (2023-02-11T14:54:00Z)
• Fairness in Matching under Uncertainty [78.39459690570531]
  algorithmic two-sided marketplaces have drawn attention to the issue of fairness in such settings. We axiomatize a notion of individual fairness in the two-sided marketplace setting which respects the uncertainty in the merits. We design a linear programming framework to find fair utility-maximizing distributions over allocations.
  arXiv  Detail & Related papers  (2023-02-08T00:30:32Z)
• Practical Approaches for Fair Learning with Multitype and Multivariate Sensitive Attributes [70.6326967720747]
  It is important to guarantee that machine learning algorithms deployed in the real world do not result in unfairness or unintended social consequences. We introduce FairCOCCO, a fairness measure built on cross-covariance operators on reproducing kernel Hilbert Spaces. We empirically demonstrate consistent improvements against state-of-the-art techniques in balancing predictive power and fairness on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-11T11:28:46Z)
• Measuring Fairness of Text Classifiers via Prediction Sensitivity [63.56554964580627]
  ACCUMULATED PREDICTION SENSITIVITY measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features. We show that the metric can be theoretically linked with a specific notion of group fairness (statistical parity) and individual fairness.
  arXiv  Detail & Related papers  (2022-03-16T15:00:33Z)
• Are There Exceptions to Goodhart's Law? On the Moral Justification of Fairness-Aware Machine Learning [14.428360876120333]
  We argue that fairness measures are particularly sensitive to Goodhart's law. We present a framework for moral reasoning about the justification of fairness metrics.
  arXiv  Detail & Related papers  (2022-02-17T09:26:39Z)
• Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research [2.6397379133308214]
  We argue that such assumptions, which are often left implicit and unexamined, lead to inconsistent conclusions. While the intended goal of this work may be to improve the fairness of machine learning models, these unexamined, implicit assumptions can in fact result in emergent unfairness.
  arXiv  Detail & Related papers  (2021-02-01T22:02:14Z)
• On the Moral Justification of Statistical Parity [0.0]
  A crucial but often neglected aspect of fairness is the question of how we justify enforcing a certain fairness metric from a moral perspective. Our aim in this paper is to consider the moral aspects associated with the statistical fairness criterion of independence.
  arXiv  Detail & Related papers  (2020-11-04T00:26:15Z)
• Two Simple Ways to Learn Individual Fairness Metrics from Data [47.6390279192406]
  Individual fairness is an intuitive definition of algorithmic fairness that addresses some of the drawbacks of group fairness. The lack of a widely accepted fair metric for many ML tasks is the main barrier to broader adoption of individual fairness. We show empirically that fair training with the learned metrics leads to improved fairness on three machine learning tasks susceptible to gender and racial biases.
  arXiv  Detail & Related papers  (2020-06-19T23:47:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.