Fairness Aware Reinforcement Learning via Proximal Policy Optimization

• URL: http://arxiv.org/abs/2502.03953v1
• Date: Thu, 06 Feb 2025 10:45:55 GMT
• Title: Fairness Aware Reinforcement Learning via Proximal Policy Optimization
• Authors: Gabriele La Malfa, Jie M. Zhang, Michael Luck, Elizabeth Black,
• Abstract summary: This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from demographic parity, counterfactual fairness, and conditional statistical parity.<n>We evaluate our approach in the Allelopathic Harvest game, a cooperative and competitive MAS focused on resource collection.
• Score: 7.061167083587786
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fairness in multi-agent systems (MAS) focuses on equitable reward distribution among agents in scenarios involving sensitive attributes such as race, gender, or socioeconomic status. This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from demographic parity, counterfactual fairness, and conditional statistical parity. The proposed method balances reward maximisation with fairness by integrating two penalty components: a retrospective component that minimises disparities in past outcomes and a prospective component that ensures fairness in future decision-making. We evaluate our approach in the Allelopathic Harvest game, a cooperative and competitive MAS focused on resource collection, where some agents possess a sensitive attribute. Experiments demonstrate that fair-PPO achieves fairer policies across all fairness metrics than classic PPO. Fairness comes at the cost of reduced rewards, namely the Price of Fairness, although agents with and without the sensitive attribute renounce comparable amounts of rewards. Additionally, the retrospective and prospective penalties effectively change the agents' behaviour and improve fairness. These findings underscore the potential of fair-PPO to address fairness challenges in MAS.

Related papers

• Understanding Accuracy-Fairness Trade-offs in Re-ranking through Elasticity in Economics [96.68144350976637]
  Fairness is an increasingly important factor in re-ranking tasks. The accuracy-fairness trade-off parallels the coupling of the commodity tax transfer process. We introduce the Elastic Fairness Curve (EF-Curve) as an evaluation framework. We also propose ElasticRank, a fair re-ranking algorithm that employs elasticity calculations to adjust inter-item distances.
  arXiv  Detail & Related papers  (2025-04-21T09:41:08Z)
• Using Protected Attributes to Consider Fairness in Multi-Agent Systems [7.061167083587786]
  Fairness in Multi-Agent Systems (MAS) depends on various factors, including the system's governing rules, the behaviour of the agents, and their characteristics. We take inspiration from the work on algorithmic fairness, which addresses bias in machine learning-based decision-making. We adapt fairness metrics from the algorithmic fairness literature to the multi-agent setting, where self-interested agents interact within an environment.
  arXiv  Detail & Related papers  (2024-10-16T08:12:01Z)
• Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation [53.285436927963865]
  We present the first comprehensive study of RAG systems that incorporate fairness-aware rankings. We find that fairness-aware retrieval frequently retains or even improves ranking effectiveness and generation quality. Our results underscore the importance of item-side fairness throughout both retrieval and generation phases.
  arXiv  Detail & Related papers  (2024-09-17T23:10:04Z)
• Fairness Incentives in Response to Unfair Dynamic Pricing [7.991187769447732]
  We design a basic simulated economy, wherein we generate corporate taxation schedules geared to incentivizing firms towards adopting fair pricing behaviours. To cover a range of possible policy scenarios, we formulate our social planner's learning problem as a multi-armed bandit, a contextual bandit and as a full reinforcement learning (RL) problem. We find that social welfare improves on that of the fairness-agnostic baseline, and approaches that of the analytically optimal fairness-aware baseline for the multi-armed and contextual bandit settings.
  arXiv  Detail & Related papers  (2024-04-22T23:12:58Z)
• Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation [46.61909578101735]
  Adversarial Policy Optimization (AdvPO) is a novel solution to the pervasive issue of reward over-optimization in Reinforcement Learning from Human Feedback. In this paper, we introduce a lightweight way to quantify uncertainties in rewards, relying solely on the last layer embeddings of the reward model.
  arXiv  Detail & Related papers  (2024-03-08T09:20:12Z)
• Causal Fairness for Outcome Control [68.12191782657437]
  We study a specific decision-making task called outcome control in which an automated system aims to optimize an outcome variable $Y$ while being fair and equitable. In this paper, we first analyze through causal lenses the notion of benefit, which captures how much a specific individual would benefit from a positive decision. We then note that the benefit itself may be influenced by the protected attribute, and propose causal tools which can be used to analyze this.
  arXiv  Detail & Related papers  (2023-06-08T09:31:18Z)
• Fair-CDA: Continuous and Directional Augmentation for Group Fairness [48.84385689186208]
  We propose a fine-grained data augmentation strategy for imposing fairness constraints. We show that group fairness can be achieved by regularizing the models on transition paths of sensitive features between groups. Our proposed method does not assume any data generative model and ensures good generalization for both accuracy and fairness.
  arXiv  Detail & Related papers  (2023-04-01T11:23:00Z)
• Proportional Fairness in Obnoxious Facility Location [70.64736616610202]
  We propose a hierarchy of distance-based proportional fairness concepts for the problem. We consider deterministic and randomized mechanisms, and compute tight bounds on the price of proportional fairness. We prove existence results for two extensions to our model.
  arXiv  Detail & Related papers  (2023-01-11T07:30:35Z)
• Fair and Optimal Classification via Post-Processing [10.163721748735801]
  This paper provides a complete characterization of the inherent tradeoff of demographic parity on classification problems. We show that the minimum error rate achievable by randomized and attribute-aware fair classifiers is given by the optimal value of a Wasserstein-barycenter problem.
  arXiv  Detail & Related papers  (2022-11-03T00:04:04Z)
• Unfairness Despite Awareness: Group-Fair Classification with Strategic Agents [37.31138342300617]
  We show that strategic agents may possess both the ability and the incentive to manipulate an observed feature vector in order to attain a more favorable outcome. We further demonstrate that both the increased selectiveness of the fair classifier, and consequently the loss of fairness, arises when performing fair learning on domains in which the advantaged group is overrepresented.
  arXiv  Detail & Related papers  (2021-12-06T02:42:43Z)
• Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning [68.25805655688876]
  Fairness in recommendation has attracted increasing attention due to bias and discrimination possibly caused by traditional recommenders. We propose a reinforcement learning based framework, FairRec, to dynamically maintain a long-term balance between accuracy and fairness in IRS. Extensive experiments validate that FairRec can improve fairness, while preserving good recommendation quality.
  arXiv  Detail & Related papers  (2021-06-25T02:02:51Z)
• Fairness for Cooperative Multi-Agent Learning with Equivariant Policies [24.92668968807012]
  We study fairness through the lens of cooperative multi-agent learning. We introduce team fairness, a group-based fairness measure for multi-agent learning. We then incorporate team fairness into policy optimization.
  arXiv  Detail & Related papers  (2021-06-10T13:17:46Z)
• Fairness, Welfare, and Equity in Personalized Pricing [88.9134799076718]
  We study the interplay of fairness, welfare, and equity considerations in personalized pricing based on customer features. We show the potential benefits of personalized pricing in two settings: pricing subsidies for an elective vaccine, and the effects of personalized interest rates on downstream outcomes in microcredit.
  arXiv  Detail & Related papers  (2020-12-21T01:01:56Z)
• Achieving Proportionality up to the Maximin Item with Indivisible Goods [14.002498730240902]
  We study the problem of fairly allocating indivisible goods and focus on the classic fairness notion of proportionality. Recent work has established that even approximate versions of proportionality (PROPx) may be impossible to achieve even for small instances. We show how to reach an allocation satisfying this notion for any instance involving up to five agents with additive valuations.
  arXiv  Detail & Related papers  (2020-09-20T19:21:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.