On the Identification of Fair Auditors to Evaluate Recommender Systems based on a Novel Non-Comparative Fairness Notion

• URL: http://arxiv.org/abs/2009.04383v1
• Date: Wed, 9 Sep 2020 16:04:41 GMT
• Title: On the Identification of Fair Auditors to Evaluate Recommender Systems based on a Novel Non-Comparative Fairness Notion
• Authors: Mukund Telukunta and Venkata Sriram Siddhardh Nadendla
• Abstract summary: Decision-support systems have been found to be discriminatory in the context of many practical deployments. We propose a new fairness notion based on the principle of non-comparative justice. We show that the proposed fairness notion also provides guarantees in terms of comparative fairness notions.
• Score: 1.116812194101501
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decision-support systems are information systems that offer support to people's decisions in various applications such as judiciary, real-estate and banking sectors. Lately, these support systems have been found to be discriminatory in the context of many practical deployments. In an attempt to evaluate and mitigate these biases, algorithmic fairness literature has been nurtured using notions of comparative justice, which relies primarily on comparing two/more individuals or groups within the society that is supported by such systems. However, such a fairness notion is not very useful in the identification of fair auditors who are hired to evaluate latent biases within decision-support systems. As a solution, we introduce a paradigm shift in algorithmic fairness via proposing a new fairness notion based on the principle of non-comparative justice. Assuming that the auditor makes fairness evaluations based on some (potentially unknown) desired properties of the decision-support system, the proposed fairness notion compares the system's outcome with that of the auditor's desired outcome. We show that the proposed fairness notion also provides guarantees in terms of comparative fairness notions by proving that any system can be deemed fair from the perspective of comparative fairness (e.g. individual fairness and statistical parity) if it is non-comparatively fair with respect to an auditor who has been deemed fair with respect to the same fairness notions. We also show that the converse holds true in the context of individual fairness. A brief discussion is also presented regarding how our fairness notion can be used to identify fair and reliable auditors, and how we can use them to quantify biases in decision-support systems.

Related papers

• Equal Confusion Fairness: Measuring Group-Based Disparities in Automated Decision Systems [5.076419064097733]
  This paper proposes a new equal confusion fairness test to check an automated decision system for fairness and a new confusion parity error to quantify the extent of any unfairness. Overall, the methods and metrics provided here may assess automated decision systems' fairness as part of a more extensive accountability assessment.
  arXiv  Detail & Related papers  (2023-07-02T04:44:19Z)
• Causal Fairness for Outcome Control [68.12191782657437]
  We study a specific decision-making task called outcome control in which an automated system aims to optimize an outcome variable $Y$ while being fair and equitable. In this paper, we first analyze through causal lenses the notion of benefit, which captures how much a specific individual would benefit from a positive decision. We then note that the benefit itself may be influenced by the protected attribute, and propose causal tools which can be used to analyze this.
  arXiv  Detail & Related papers  (2023-06-08T09:31:18Z)
• A Survey on Fairness-aware Recommender Systems [59.23208133653637]
  We present concepts of fairness in different recommendation scenarios, comprehensively categorize current advances, and introduce typical methods to promote fairness in different stages of recommender systems. Next, we delve into the significant influence that fairness-aware recommender systems exert on real-world industrial applications.
  arXiv  Detail & Related papers  (2023-06-01T07:08:22Z)
• Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models [73.24381010980606]
  This study examines issues of algorithmic fairness in the context of systems that inform tax audit selection by the IRS. We show how the use of more flexible machine learning methods for selecting audits may affect vertical equity. Our results have implications for the design of algorithmic tools across the public sector.
  arXiv  Detail & Related papers  (2022-06-20T16:27:06Z)
• Fairness Score and Process Standardization: Framework for Fairness Certification in Artificial Intelligence Systems [0.4297070083645048]
  We propose a novel Fairness Score to measure the fairness of a data-driven AI system. It will also provide a framework to operationalise the concept of fairness and facilitate the commercial deployment of such systems.
  arXiv  Detail & Related papers  (2022-01-10T15:45:12Z)
• Non-Comparative Fairness for Human-Auditing and Its Relation to Traditional Fairness Notions [1.8275108630751837]
  This paper proposes a new fairness notion based on the principle of non-comparative justice. We show that any MLS can be deemed fair from the perspective of comparative fairness. We also show that the converse holds true in the context of individual fairness.
  arXiv  Detail & Related papers  (2021-06-29T20:05:22Z)
• Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning [68.25805655688876]
  Fairness in recommendation has attracted increasing attention due to bias and discrimination possibly caused by traditional recommenders. We propose a reinforcement learning based framework, FairRec, to dynamically maintain a long-term balance between accuracy and fairness in IRS. Extensive experiments validate that FairRec can improve fairness, while preserving good recommendation quality.
  arXiv  Detail & Related papers  (2021-06-25T02:02:51Z)
• Equality before the Law: Legal Judgment Consistency Analysis for Fairness [55.91612739713396]
  In this paper, we propose an evaluation metric for judgment inconsistency, Legal Inconsistency Coefficient (LInCo) We simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups. We employ LInCo to explore the inconsistency in real cases and come to the following observations: (1) Both regional and gender inconsistency exist in the legal system, but gender inconsistency is much less than regional inconsistency.
  arXiv  Detail & Related papers  (2021-03-25T14:28:00Z)
• "And the Winner Is...": Dynamic Lotteries for Multi-group Fairness-Aware Recommendation [37.35485045640196]
  We argue that the previous literature has been based on simple, uniform and often uni-dimensional notions of fairness assumptions. We explicitly represent the design decisions that enter into the trade-off between accuracy and fairness across multiply-defined and intersecting protected groups. We formulate lottery-based mechanisms for choosing between fairness concerns, and demonstrate their performance in two recommendation domains.
  arXiv  Detail & Related papers  (2020-09-05T20:15:14Z)
• Algorithmic Decision Making with Conditional Fairness [48.76267073341723]
  We define conditional fairness as a more sound fairness metric by conditioning on the fairness variables. We propose a Derivable Conditional Fairness Regularizer (DCFR) to track the trade-off between precision and fairness of algorithmic decision making.
  arXiv  Detail & Related papers  (2020-06-18T12:56:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.