Fairness Aware Reinforcement Learning via Proximal Policy Optimization

• URL: http://arxiv.org/abs/2502.03953v1
• Date: Thu, 06 Feb 2025 10:45:55 GMT
• Title: Fairness Aware Reinforcement Learning via Proximal Policy Optimization
• Authors: Gabriele La Malfa, Jie M. Zhang, Michael Luck, Elizabeth Black,
• Abstract summary: This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from demographic parity, counterfactual fairness, and conditional statistical parity. We evaluate our approach in the Allelopathic Harvest game, a cooperative and competitive MAS focused on resource collection.
• Score: 7.061167083587786
• License:
• Abstract: Fairness in multi-agent systems (MAS) focuses on equitable reward distribution among agents in scenarios involving sensitive attributes such as race, gender, or socioeconomic status. This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from demographic parity, counterfactual fairness, and conditional statistical parity. The proposed method balances reward maximisation with fairness by integrating two penalty components: a retrospective component that minimises disparities in past outcomes and a prospective component that ensures fairness in future decision-making. We evaluate our approach in the Allelopathic Harvest game, a cooperative and competitive MAS focused on resource collection, where some agents possess a sensitive attribute. Experiments demonstrate that fair-PPO achieves fairer policies across all fairness metrics than classic PPO. Fairness comes at the cost of reduced rewards, namely the Price of Fairness, although agents with and without the sensitive attribute renounce comparable amounts of rewards. Additionally, the retrospective and prospective penalties effectively change the agents' behaviour and improve fairness. These findings underscore the potential of fair-PPO to address fairness challenges in MAS.

Related papers

• Using Protected Attributes to Consider Fairness in Multi-Agent Systems [7.061167083587786]
  Fairness in Multi-Agent Systems (MAS) depends on various factors, including the system's governing rules, the behaviour of the agents, and their characteristics. We take inspiration from the work on algorithmic fairness, which addresses bias in machine learning-based decision-making. We adapt fairness metrics from the algorithmic fairness literature to the multi-agent setting, where self-interested agents interact within an environment.
  arXiv  Detail & Related papers  (2024-10-16T08:12:01Z)
• Fairness Incentives in Response to Unfair Dynamic Pricing [7.991187769447732]
  We design a basic simulated economy, wherein we generate corporate taxation schedules geared to incentivizing firms towards adopting fair pricing behaviours. To cover a range of possible policy scenarios, we formulate our social planner's learning problem as a multi-armed bandit, a contextual bandit and as a full reinforcement learning (RL) problem. We find that social welfare improves on that of the fairness-agnostic baseline, and approaches that of the analytically optimal fairness-aware baseline for the multi-armed and contextual bandit settings.
  arXiv  Detail & Related papers  (2024-04-22T23:12:58Z)
• Mimicking Better by Matching the Approximate Action Distribution [48.95048003354255]
  We introduce MAAD, a novel, sample-efficient on-policy algorithm for Imitation Learning from Observations. We show that it requires considerable fewer interactions to achieve expert performance, outperforming current state-of-the-art on-policy methods.
  arXiv  Detail & Related papers  (2023-06-16T12:43:47Z)
• Causal Fairness for Outcome Control [68.12191782657437]
  We study a specific decision-making task called outcome control in which an automated system aims to optimize an outcome variable $Y$ while being fair and equitable. In this paper, we first analyze through causal lenses the notion of benefit, which captures how much a specific individual would benefit from a positive decision. We then note that the benefit itself may be influenced by the protected attribute, and propose causal tools which can be used to analyze this.
  arXiv  Detail & Related papers  (2023-06-08T09:31:18Z)
• Fair-CDA: Continuous and Directional Augmentation for Group Fairness [48.84385689186208]
  We propose a fine-grained data augmentation strategy for imposing fairness constraints. We show that group fairness can be achieved by regularizing the models on transition paths of sensitive features between groups. Our proposed method does not assume any data generative model and ensures good generalization for both accuracy and fairness.
  arXiv  Detail & Related papers  (2023-04-01T11:23:00Z)
• Proportional Fairness in Obnoxious Facility Location [70.64736616610202]
  We propose a hierarchy of distance-based proportional fairness concepts for the problem. We consider deterministic and randomized mechanisms, and compute tight bounds on the price of proportional fairness. We prove existence results for two extensions to our model.
  arXiv  Detail & Related papers  (2023-01-11T07:30:35Z)
• Unfairness Despite Awareness: Group-Fair Classification with Strategic Agents [37.31138342300617]
  We show that strategic agents may possess both the ability and the incentive to manipulate an observed feature vector in order to attain a more favorable outcome. We further demonstrate that both the increased selectiveness of the fair classifier, and consequently the loss of fairness, arises when performing fair learning on domains in which the advantaged group is overrepresented.
  arXiv  Detail & Related papers  (2021-12-06T02:42:43Z)
• Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning [68.25805655688876]
  Fairness in recommendation has attracted increasing attention due to bias and discrimination possibly caused by traditional recommenders. We propose a reinforcement learning based framework, FairRec, to dynamically maintain a long-term balance between accuracy and fairness in IRS. Extensive experiments validate that FairRec can improve fairness, while preserving good recommendation quality.
  arXiv  Detail & Related papers  (2021-06-25T02:02:51Z)
• Fairness for Cooperative Multi-Agent Learning with Equivariant Policies [24.92668968807012]
  We study fairness through the lens of cooperative multi-agent learning. We introduce team fairness, a group-based fairness measure for multi-agent learning. We then incorporate team fairness into policy optimization.
  arXiv  Detail & Related papers  (2021-06-10T13:17:46Z)
• Fairness, Welfare, and Equity in Personalized Pricing [88.9134799076718]
  We study the interplay of fairness, welfare, and equity considerations in personalized pricing based on customer features. We show the potential benefits of personalized pricing in two settings: pricing subsidies for an elective vaccine, and the effects of personalized interest rates on downstream outcomes in microcredit.
  arXiv  Detail & Related papers  (2020-12-21T01:01:56Z)
• Achieving Proportionality up to the Maximin Item with Indivisible Goods [14.002498730240902]
  We study the problem of fairly allocating indivisible goods and focus on the classic fairness notion of proportionality. Recent work has established that even approximate versions of proportionality (PROPx) may be impossible to achieve even for small instances. We show how to reach an allocation satisfying this notion for any instance involving up to five agents with additive valuations.
  arXiv  Detail & Related papers  (2020-09-20T19:21:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.