Evaluating AI Group Fairness: a Fuzzy Logic Perspective

• URL: http://arxiv.org/abs/2406.18939v1
• Date: Thu, 27 Jun 2024 07:11:48 GMT
• Title: Evaluating AI Group Fairness: a Fuzzy Logic Perspective
• Authors: Emmanouil Krasanakis, Symeon Papadopoulos,
• Abstract summary: What constitutes group fairness depends on who is asked and the social context, whereas definitions are often relaxed to accept small deviations from the statistical constraints they set out to impose. Here we decouple definitions of group fairness from the context and from relaxation-related uncertainty by expressing them in the axiomatic system of Basic fuzzy Logic. We show that commonly held propositions standardize the resulting mathematical formulas and we transcribe logic and truth value choices to layperson terms.
• Score: 9.876903282565976
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence systems often address fairness concerns by evaluating and mitigating measures of group discrimination, for example that indicate biases against certain genders or races. However, what constitutes group fairness depends on who is asked and the social context, whereas definitions are often relaxed to accept small deviations from the statistical constraints they set out to impose. Here we decouple definitions of group fairness both from the context and from relaxation-related uncertainty by expressing them in the axiomatic system of Basic fuzzy Logic (BL) with loosely understood predicates, like encountering group members. We then evaluate the definitions in subclasses of BL, such as Product or Lukasiewicz logics. Evaluation produces continuous instead of binary truth values by choosing the logic subclass and truth values for predicates that reflect uncertain context-specific beliefs, such as stakeholder opinions gathered through questionnaires. Internally, it follows logic-specific rules to compute the truth values of definitions. We show that commonly held propositions standardize the resulting mathematical formulas and we transcribe logic and truth value choices to layperson terms, so that anyone can answer them. We also use our framework to study several literature definitions of algorithmic fairness, for which we rationalize previous expedient practices that are non-probabilistic and show how to re-interpret their formulas and parameters in new contexts.

Related papers

• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism [62.571419297164645]
  This paper provides a systematic overview of prior works on the logical reasoning ability of large language models for analyzing categorical syllogisms. We first investigate all the possible variations for the categorical syllogisms from a purely logical perspective. We then examine the underlying configurations (i.e., mood and figure) tested by the existing datasets.
  arXiv  Detail & Related papers  (2024-06-26T21:17:20Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Distributive Justice as the Foundational Premise of Fair ML: Unification, Extension, and Interpretation of Group Fairness Metrics [0.0]
  Group fairness metrics are an established way of assessing the fairness of prediction-based decision-making systems. We propose a comprehensive framework for group fairness metrics, which links them to more theories of distributive justice.
  arXiv  Detail & Related papers  (2022-06-06T20:44:02Z)
• Is calibration a fairness requirement? An argument from the point of view of moral philosophy and decision theory [0.0]
  We argue that a violation of group calibration may be unfair in some cases, but not unfair in others. This is in line with claims already advanced in the literature, that algorithmic fairness should be defined in a way that is sensitive to context.
  arXiv  Detail & Related papers  (2022-05-11T14:03:33Z)
• Measuring Fairness of Text Classifiers via Prediction Sensitivity [63.56554964580627]
  ACCUMULATED PREDICTION SENSITIVITY measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features. We show that the metric can be theoretically linked with a specific notion of group fairness (statistical parity) and individual fairness.
  arXiv  Detail & Related papers  (2022-03-16T15:00:33Z)
• Fairness Through Counterfactual Utilities [0.0]
  Group fairness definitions such as Demographic Parity and Equal Opportunity make assumptions about the underlying decision-problem that restrict them to classification problems. We provide a generalized set of group fairness definitions that unambiguously extend to all machine learning environments.
  arXiv  Detail & Related papers  (2021-08-11T16:51:27Z)
• Fairness for Image Generation with Uncertain Sensitive Attributes [97.81354305427871]
  This work tackles the issue of fairness in the context of generative procedures, such as image super-resolution. While traditional group fairness definitions are typically defined with respect to specified protected groups, we emphasize that there are no ground truth identities. We show that the natural extension of demographic parity is strongly dependent on the grouping, and emphimpossible to achieve obliviously.
  arXiv  Detail & Related papers  (2021-06-23T06:17:17Z)
• Foundations of Reasoning with Uncertainty via Real-valued Logics [70.43924776071616]
  We give a sound and strongly complete axiomatization that can be parametrized to cover essentially every real-valued logic. Our class of sentences are very rich, and each describes a set of possible real values for a collection of formulas of the real-valued logic.
  arXiv  Detail & Related papers  (2020-08-06T02:13:11Z)
• Abstracting Fairness: Oracles, Metrics, and Interpretability [21.59432019966861]
  We examine what can be learned from a fairness oracle equipped with an underlying understanding of true'' fairness. Our results have implications for interpretablity -- a highly desired but poorly defined property of classification systems.
  arXiv  Detail & Related papers  (2020-04-04T03:14:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.