Rethinking Fairness for Human-AI Collaboration

• URL: http://arxiv.org/abs/2310.03647v1
• Date: Thu, 5 Oct 2023 16:21:42 GMT
• Title: Rethinking Fairness for Human-AI Collaboration
• Authors: Haosen Ge, Hamsa Bastani, Osbert Bastani
• Abstract summary: We propose a simple optimization strategy to identify the best performance-improving compliance-robustly fair policy. It may be infeasible to design algorithmic recommendations that are simultaneously fair in isolation, compliance-robustly fair, and more accurate than the human policy.
• Score: 32.969050978497066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing approaches to algorithmic fairness aim to ensure equitable outcomes if human decision-makers comply perfectly with algorithmic decisions. However, perfect compliance with the algorithm is rarely a reality or even a desirable outcome in human-AI collaboration. Yet, recent studies have shown that selective compliance with fair algorithms can amplify discrimination relative to the prior human policy. As a consequence, ensuring equitable outcomes requires fundamentally different algorithmic design principles that ensure robustness to the decision-maker's (a priori unknown) compliance pattern. We define the notion of compliance-robustly fair algorithmic recommendations that are guaranteed to (weakly) improve fairness in decisions, regardless of the human's compliance pattern. We propose a simple optimization strategy to identify the best performance-improving compliance-robustly fair policy. However, we show that it may be infeasible to design algorithmic recommendations that are simultaneously fair in isolation, compliance-robustly fair, and more accurate than the human policy; thus, if our goal is to improve the equity and accuracy of human-AI collaboration, it may not be desirable to enforce traditional fairness constraints.

Related papers

• Peer-induced Fairness: A Causal Approach for Algorithmic Fairness Auditing [0.0]
  The European Union's Artificial Intelligence Act takes effect on 1 August 2024. High-risk AI applications must adhere to stringent transparency and fairness standards. We propose a novel framework, which combines the strengths of counterfactual fairness and peer comparison strategy.
  arXiv  Detail & Related papers  (2024-08-05T15:35:34Z)
• Joint Demonstration and Preference Learning Improves Policy Alignment with Human Feedback [58.049113055986375]
  We develop a single stage approach named Alignment with Integrated Human Feedback (AIHF) to train reward models and the policy. The proposed approach admits a suite of efficient algorithms, which can easily reduce to, and leverage, popular alignment algorithms. We demonstrate the efficiency of the proposed solutions with extensive experiments involving alignment problems in LLMs and robotic control problems in MuJoCo.
  arXiv  Detail & Related papers  (2024-06-11T01:20:53Z)
• Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
  We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences. To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model. Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
  arXiv  Detail & Related papers  (2024-05-26T05:38:50Z)
• Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedback [57.6775169085215]
  Risk-sensitive reinforcement learning aims to optimize policies that balance the expected reward and risk. We present a novel framework that employs an Iterated Conditional Value-at-Risk (CVaR) objective under both linear and general function approximations. We propose provably sample-efficient algorithms for this Iterated CVaR RL and provide rigorous theoretical analysis.
  arXiv  Detail & Related papers  (2023-07-06T08:14:54Z)
• Robustness Implies Fairness in Casual Algorithmic Recourse [13.86376549140248]
  Algorithmic recourse aims to disclose the inner workings of the black-box decision process in situations where decisions have significant consequences. To ensure an effective remedy, suggested interventions must not only be low-cost but also robust and fair. This study explores the concept of individual fairness and adversarial robustness in causal algorithmic recourse.
  arXiv  Detail & Related papers  (2023-02-07T13:40:56Z)
• Beyond Incompatibility: Trade-offs between Mutually Exclusive Fairness Criteria in Machine Learning and Law [2.959308758321417]
  We present a novel algorithm (FAir Interpolation Method: FAIM) for continuously interpolating between three fairness criteria. We demonstrate the effectiveness of our algorithm when applied to synthetic data, the COMPAS data set, and a new, real-world data set from the e-commerce sector.
  arXiv  Detail & Related papers  (2022-12-01T12:47:54Z)
• Active Fairness Auditing [22.301071549943064]
  We study query-based auditing algorithms that can estimate the demographic parity of ML models in a query-efficient manner. We propose an optimal deterministic algorithm, as well as a practical randomized, oracle-efficient algorithm with comparable guarantees. Our first exploration of active fairness estimation aims to put AI governance on firmer theoretical foundations.
  arXiv  Detail & Related papers  (2022-06-16T21:12:00Z)
• Understanding Relations Between Perception of Fairness and Trust in Algorithmic Decision Making [8.795591344648294]
  We aim to understand the relationship between induced algorithmic fairness and its perception in humans. We also study how does induced algorithmic fairness affects user trust in algorithmic decision making.
  arXiv  Detail & Related papers  (2021-09-29T11:00:39Z)
• On the Optimality of Batch Policy Optimization Algorithms [106.89498352537682]
  Batch policy optimization considers leveraging existing data for policy construction before interacting with an environment. We show that any confidence-adjusted index algorithm is minimax optimal, whether it be optimistic, pessimistic or neutral. We introduce a new weighted-minimax criterion that considers the inherent difficulty of optimal value prediction.
  arXiv  Detail & Related papers  (2021-04-06T05:23:20Z)
• Beyond Individual and Group Fairness [90.4666341812857]
  We present a new data-driven model of fairness that is guided by the unfairness complaints received by the system. Our model supports multiple fairness criteria and takes into account their potential incompatibilities.
  arXiv  Detail & Related papers  (2020-08-21T14:14:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.