Cooperation and Fairness in Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2410.14916v1
• Date: Sat, 19 Oct 2024 00:10:52 GMT
• Title: Cooperation and Fairness in Multi-Agent Reinforcement Learning
• Authors: Jasmine Jerry Aloor, Siddharth Nayak, Sydney Dolan, Hamsa Balakrishnan,
• Abstract summary: In resource-constrained environments of mobility and transportation systems, efficiency may be achieved at the expense of fairness. We consider the problem of fair multi-agent navigation for a group of decentralized agents using multi-agent reinforcement learning (MARL) We find that our model yields a 14% improvement in efficiency and a 5% improvement in fairness over a baseline trained using random assignments.
• Score: 6.164771707307928
• License:
• Abstract: Multi-agent systems are trained to maximize shared cost objectives, which typically reflect system-level efficiency. However, in the resource-constrained environments of mobility and transportation systems, efficiency may be achieved at the expense of fairness -- certain agents may incur significantly greater costs or lower rewards compared to others. Tasks could be distributed inequitably, leading to some agents receiving an unfair advantage while others incur disproportionately high costs. It is important to consider the tradeoffs between efficiency and fairness. We consider the problem of fair multi-agent navigation for a group of decentralized agents using multi-agent reinforcement learning (MARL). We consider the reciprocal of the coefficient of variation of the distances traveled by different agents as a measure of fairness and investigate whether agents can learn to be fair without significantly sacrificing efficiency (i.e., increasing the total distance traveled). We find that by training agents using min-max fair distance goal assignments along with a reward term that incentivizes fairness as they move towards their goals, the agents (1) learn a fair assignment of goals and (2) achieve almost perfect goal coverage in navigation scenarios using only local observations. For goal coverage scenarios, we find that, on average, our model yields a 14% improvement in efficiency and a 5% improvement in fairness over a baseline trained using random assignments. Furthermore, an average of 21% improvement in fairness can be achieved compared to a model trained on optimally efficient assignments; this increase in fairness comes at the expense of only a 7% decrease in efficiency. Finally, we extend our method to environments in which agents must complete coverage tasks in prescribed formations and show that it is possible to do so without tailoring the models to specific formation shapes.

Related papers

• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process. We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• FairLoRA: Unpacking Bias Mitigation in Vision Models with Fairness-Driven Low-Rank Adaptation [3.959853359438669]
  We introduce FairLoRA, a novel fairness-specific regularizer for Low Rank Adaptation (LoRA) Our results demonstrate that the need for higher ranks to mitigate bias is not universal; it depends on factors such as the pre-trained model, dataset, and task.
  arXiv  Detail & Related papers  (2024-10-22T18:50:36Z)
• Fair Few-shot Learning with Auxiliary Sets [53.30014767684218]
  In many machine learning (ML) tasks, only very few labeled data samples can be collected, which can lead to inferior fairness performance. In this paper, we define the fairness-aware learning task with limited training samples as the emphfair few-shot learning problem. We devise a novel framework that accumulates fairness-aware knowledge across different meta-training tasks and then generalizes the learned knowledge to meta-test tasks.
  arXiv  Detail & Related papers  (2023-08-28T06:31:37Z)
• Using Simple Incentives to Improve Two-Sided Fairness in Ridesharing Systems [27.34946988130242]
  We propose a simple incentive-based fairness scheme that can be implemented online as a part of this ILP formulation. We show how these fairness incentives can be formulated for two distinct use cases for passenger groups and driver fairness.
  arXiv  Detail & Related papers  (2023-03-25T02:24:27Z)
• Distributional Reward Estimation for Effective Multi-Agent Deep Reinforcement Learning [19.788336796981685]
  We propose a novel Distributional Reward Estimation framework for effective Multi-Agent Reinforcement Learning (DRE-MARL) Our main idea is to design the multi-action-branch reward estimation and policy-weighted reward aggregation for stabilized training. The superiority of the DRE-MARL is demonstrated using benchmark multi-agent scenarios, compared with the SOTA baselines in terms of both effectiveness and robustness.
  arXiv  Detail & Related papers  (2022-10-14T08:31:45Z)
• Improving Robust Fairness via Balance Adversarial Training [51.67643171193376]
  Adversarial training (AT) methods are effective against adversarial attacks, yet they introduce severe disparity of accuracy and robustness between different classes. We propose Adversarial Training (BAT) to address the robust fairness problem.
  arXiv  Detail & Related papers  (2022-09-15T14:44:48Z)
• Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience [89.30876995059168]
  inverse reinforcement learning (IRL) -- inferring the reward function of an agent from observing its behavior. This paper addresses the problem of IRL -- inferring the reward function of an agent from observing its behavior.
  arXiv  Detail & Related papers  (2022-08-09T17:29:49Z)
• Decentralized scheduling through an adaptive, trading-based multi-agent system [1.7403133838762448]
  In multi-agent reinforcement learning systems, the actions of one agent can have a negative impact on the rewards of other agents. This work applies a trading approach to a simulated scheduling environment, where the agents are responsible for the assignment of incoming jobs to compute cores. The agents can trade the usage right of computational cores to process high-priority, high-reward jobs faster than low-priority, low-reward jobs.
  arXiv  Detail & Related papers  (2022-07-05T13:50:18Z)
• Towards Equal Opportunity Fairness through Adversarial Learning [64.45845091719002]
  Adversarial training is a common approach for bias mitigation in natural language processing. We propose an augmented discriminator for adversarial training, which takes the target class as input to create richer features.
  arXiv  Detail & Related papers  (2022-03-12T02:22:58Z)
• Fairness for Cooperative Multi-Agent Learning with Equivariant Policies [24.92668968807012]
  We study fairness through the lens of cooperative multi-agent learning. We introduce team fairness, a group-based fairness measure for multi-agent learning. We then incorporate team fairness into policy optimization.
  arXiv  Detail & Related papers  (2021-06-10T13:17:46Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.