Fairness Aware Reinforcement Learning via Proximal Policy Optimization

• URL: http://arxiv.org/abs/2502.03953v2
• Date: Tue, 02 Sep 2025 10:01:32 GMT
• Title: Fairness Aware Reinforcement Learning via Proximal Policy Optimization
• Authors: Gabriele La Malfa, Jie M. Zhang, Michael Luck, Elizabeth Black,
• Abstract summary: This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from a fairness definition.<n>We evaluate our approach in two games: the Allelopathic Harvest, a cooperative and competitive MAS focused on resource collection, and HospitalSim, a hospital simulation.
• Score: 9.676752763382476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fairness in multi-agent systems (MAS) focuses on equitable reward distribution among agents in scenarios involving sensitive attributes such as race, gender, or socioeconomic status. This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from a fairness definition such as demographic parity, counterfactual fairness, or conditional statistical parity. The proposed method, which we call Fair-PPO, balances reward maximisation with fairness by integrating two penalty components: a retrospective component that minimises disparities in past outcomes and a prospective component that ensures fairness in future decision-making. We evaluate our approach in two games: the Allelopathic Harvest, a cooperative and competitive MAS focused on resource collection, where some agents possess a sensitive attribute, and HospitalSim, a hospital simulation, in which agents coordinate the operations of hospital patients with different mobility and priority needs. Experiments show that Fair-PPO achieves fairer policies than PPO across the fairness metrics and, through the retrospective and prospective penalty components, reveals a wide spectrum of strategies to improve fairness; at the same time, its performance pairs with that of state-of-the-art fair reinforcement-learning algorithms. Fairness comes at the cost of reduced efficiency, but does not compromise equality among the overall population (Gini index). These findings underscore the potential of Fair-PPO to address fairness challenges in MAS.

Related papers

• Fairness Aware Reward Optimization [78.85867531002346]
  We introduce Fairness Aware Reward Optimization (Faro), an in-processing framework that trains reward models under demographic parity, equalized odds, or counterfactual fairness constraints.<n>We provide the first theoretical analysis of reward-level fairness in LLM alignment.<n>Faro significantly reduces bias and harmful generations while maintaining or improving model quality.
  arXiv  Detail & Related papers  (2026-02-08T03:35:49Z)
• A Unifying Human-Centered AI Fairness Framework [2.9385229328767988]
  We introduce a unifying human-centered fairness framework that covers eight distinct fairness metrics.<n>Rather than privileging a single fairness notion, the framework enables stakeholders to assign weights across multiple fairness objectives.<n>We show that adjusting weights reveals nuanced trade-offs between different fairness metrics.
  arXiv  Detail & Related papers  (2025-12-07T17:52:38Z)
• Fair-GNE : Generalized Nash Equilibrium-Seeking Fairness in Multiagent Healthcare Automation [0.0]
  Existing multi-agent reinforcement learning approaches steer fairness by shaping reward through post hoc orchestrations.<n>We address this shortcoming with a learning enabled optimization scheme among self-interested decision makers.<n>Our results communicate our formulations, evaluation metrics, and equilibrium-seeking innovations in large multi-agent learning-based healthcare systems.
  arXiv  Detail & Related papers  (2025-11-18T04:48:50Z)
• A General Incentives-Based Framework for Fairness in Multi-agent Resource Allocation [4.930376365020355]
  We introduce the General Incentives-based Framework for Fairness (GIFF)<n>GIFF is a novel approach for fair multi-agent resource allocation that infers fair decision-making from standard value functions.
  arXiv  Detail & Related papers  (2025-10-30T17:37:51Z)
• FairPOT: Balancing AUC Performance and Fairness with Proportional Optimal Transport [9.47500733579604]
  In high-stakes domains such as healthcare, fairness is often evaluated over risk scores rather than binary outcomes.<n>We propose Fair Proportional Optimal Transport (FairPOT), a novel, model-agnostic post-processing framework.<n>We show that FairPOT consistently outperforms existing post-processing techniques in both global and partial AUC scenarios.
  arXiv  Detail & Related papers  (2025-08-05T22:13:08Z)
• Towards Reward Fairness in RLHF: From a Resource Allocation Perspective [16.82198859401237]
  This paper collectively defines the various biases present in rewards as the problem of reward unfairness.<n>We propose a bias-agnostic method to address the issue of reward fairness from a resource allocation perspective.
  arXiv  Detail & Related papers  (2025-05-29T11:12:00Z)
• Understanding Accuracy-Fairness Trade-offs in Re-ranking through Elasticity in Economics [96.68144350976637]
  Fairness is an increasingly important factor in re-ranking tasks. The accuracy-fairness trade-off parallels the coupling of the commodity tax transfer process. We introduce the Elastic Fairness Curve (EF-Curve) as an evaluation framework. We also propose ElasticRank, a fair re-ranking algorithm that employs elasticity calculations to adjust inter-item distances.
  arXiv  Detail & Related papers  (2025-04-21T09:41:08Z)
• Using Protected Attributes to Consider Fairness in Multi-Agent Systems [7.061167083587786]
  Fairness in Multi-Agent Systems (MAS) depends on various factors, including the system's governing rules, the behaviour of the agents, and their characteristics. We take inspiration from the work on algorithmic fairness, which addresses bias in machine learning-based decision-making. We adapt fairness metrics from the algorithmic fairness literature to the multi-agent setting, where self-interested agents interact within an environment.
  arXiv  Detail & Related papers  (2024-10-16T08:12:01Z)
• Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation [53.285436927963865]
  We present the first comprehensive study of RAG systems that incorporate fairness-aware rankings. We find that fairness-aware retrieval frequently retains or even improves ranking effectiveness and generation quality. Our results underscore the importance of item-side fairness throughout both retrieval and generation phases.
  arXiv  Detail & Related papers  (2024-09-17T23:10:04Z)
• Fairness Incentives in Response to Unfair Dynamic Pricing [7.991187769447732]
  We design a basic simulated economy, wherein we generate corporate taxation schedules geared to incentivizing firms towards adopting fair pricing behaviours. To cover a range of possible policy scenarios, we formulate our social planner's learning problem as a multi-armed bandit, a contextual bandit and as a full reinforcement learning (RL) problem. We find that social welfare improves on that of the fairness-agnostic baseline, and approaches that of the analytically optimal fairness-aware baseline for the multi-armed and contextual bandit settings.
  arXiv  Detail & Related papers  (2024-04-22T23:12:58Z)
• Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation [46.61909578101735]
  Adversarial Policy Optimization (AdvPO) is a novel solution to the pervasive issue of reward over-optimization in Reinforcement Learning from Human Feedback. In this paper, we introduce a lightweight way to quantify uncertainties in rewards, relying solely on the last layer embeddings of the reward model.
  arXiv  Detail & Related papers  (2024-03-08T09:20:12Z)
• Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
  We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
  arXiv  Detail & Related papers  (2023-11-03T21:19:59Z)
• Causal Fairness for Outcome Control [68.12191782657437]
  We study a specific decision-making task called outcome control in which an automated system aims to optimize an outcome variable $Y$ while being fair and equitable. In this paper, we first analyze through causal lenses the notion of benefit, which captures how much a specific individual would benefit from a positive decision. We then note that the benefit itself may be influenced by the protected attribute, and propose causal tools which can be used to analyze this.
  arXiv  Detail & Related papers  (2023-06-08T09:31:18Z)
• Fair-CDA: Continuous and Directional Augmentation for Group Fairness [48.84385689186208]
  We propose a fine-grained data augmentation strategy for imposing fairness constraints. We show that group fairness can be achieved by regularizing the models on transition paths of sensitive features between groups. Our proposed method does not assume any data generative model and ensures good generalization for both accuracy and fairness.
  arXiv  Detail & Related papers  (2023-04-01T11:23:00Z)
• Proportional Fairness in Obnoxious Facility Location [70.64736616610202]
  We propose a hierarchy of distance-based proportional fairness concepts for the problem. We consider deterministic and randomized mechanisms, and compute tight bounds on the price of proportional fairness. We prove existence results for two extensions to our model.
  arXiv  Detail & Related papers  (2023-01-11T07:30:35Z)
• Fair and Optimal Classification via Post-Processing [10.163721748735801]
  This paper provides a complete characterization of the inherent tradeoff of demographic parity on classification problems. We show that the minimum error rate achievable by randomized and attribute-aware fair classifiers is given by the optimal value of a Wasserstein-barycenter problem.
  arXiv  Detail & Related papers  (2022-11-03T00:04:04Z)
• Fair Machine Learning in Healthcare: A Review [90.22219142430146]
  We analyze the intersection of fairness in machine learning and healthcare disparities. We provide a critical review of the associated fairness metrics from a machine learning standpoint. We propose several new research directions that hold promise for developing ethical and equitable ML applications in healthcare.
  arXiv  Detail & Related papers  (2022-06-29T04:32:10Z)
• Monotonic Improvement Guarantees under Non-stationarity for Decentralized PPO [66.5384483339413]
  We present a new monotonic improvement guarantee for optimizing decentralized policies in cooperative Multi-Agent Reinforcement Learning (MARL) We show that a trust region constraint can be effectively enforced in a principled way by bounding independent ratios based on the number of agents in training.
  arXiv  Detail & Related papers  (2022-01-31T20:39:48Z)
• Unfairness Despite Awareness: Group-Fair Classification with Strategic Agents [37.31138342300617]
  We show that strategic agents may possess both the ability and the incentive to manipulate an observed feature vector in order to attain a more favorable outcome. We further demonstrate that both the increased selectiveness of the fair classifier, and consequently the loss of fairness, arises when performing fair learning on domains in which the advantaged group is overrepresented.
  arXiv  Detail & Related papers  (2021-12-06T02:42:43Z)
• Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning [68.25805655688876]
  Fairness in recommendation has attracted increasing attention due to bias and discrimination possibly caused by traditional recommenders. We propose a reinforcement learning based framework, FairRec, to dynamically maintain a long-term balance between accuracy and fairness in IRS. Extensive experiments validate that FairRec can improve fairness, while preserving good recommendation quality.
  arXiv  Detail & Related papers  (2021-06-25T02:02:51Z)
• Fairness for Cooperative Multi-Agent Learning with Equivariant Policies [24.92668968807012]
  We study fairness through the lens of cooperative multi-agent learning. We introduce team fairness, a group-based fairness measure for multi-agent learning. We then incorporate team fairness into policy optimization.
  arXiv  Detail & Related papers  (2021-06-10T13:17:46Z)
• Multi-Stage Decentralized Matching Markets: Uncertain Preferences and Strategic Behaviors [91.3755431537592]
  This article develops a framework for learning optimal strategies in real-world matching markets. We show that there exists a welfare-versus-fairness trade-off that is characterized by the uncertainty level of acceptance. We prove that participants can be better off with multi-stage matching compared to single-stage matching.
  arXiv  Detail & Related papers  (2021-02-13T19:25:52Z)
• Fairness, Welfare, and Equity in Personalized Pricing [88.9134799076718]
  We study the interplay of fairness, welfare, and equity considerations in personalized pricing based on customer features. We show the potential benefits of personalized pricing in two settings: pricing subsidies for an elective vaccine, and the effects of personalized interest rates on downstream outcomes in microcredit.
  arXiv  Detail & Related papers  (2020-12-21T01:01:56Z)
• Achieving Proportionality up to the Maximin Item with Indivisible Goods [14.002498730240902]
  We study the problem of fairly allocating indivisible goods and focus on the classic fairness notion of proportionality. Recent work has established that even approximate versions of proportionality (PROPx) may be impossible to achieve even for small instances. We show how to reach an allocation satisfying this notion for any instance involving up to five agents with additive valuations.
  arXiv  Detail & Related papers  (2020-09-20T19:21:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.