Group Fairness in Multi-Task Reinforcement Learning

• URL: http://arxiv.org/abs/2503.07817v1
• Date: Mon, 10 Mar 2025 19:59:59 GMT
• Title: Group Fairness in Multi-Task Reinforcement Learning
• Authors: Kefan Song, Runnan Jiang, Rohan Chandra, Shangtong Zhang,
• Abstract summary: We introduce a novel formulation of multi-task group fairness in Reinforcement Learning (RL)<n>We propose a constrained optimization algorithm that explicitly enforces fairness constraints across multiple tasks simultaneously.<n>Our results show that the proposed algorithm achieves smaller fairness gaps while maintaining comparable returns across different demographic groups and tasks.
• Score: 17.98928508837315
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper addresses a critical societal consideration in the application of Reinforcement Learning (RL): ensuring equitable outcomes across different demographic groups in multi-task settings. While previous work has explored fairness in single-task RL, many real-world applications are multi-task in nature and require policies to maintain fairness across all tasks. We introduce a novel formulation of multi-task group fairness in RL and propose a constrained optimization algorithm that explicitly enforces fairness constraints across multiple tasks simultaneously. We have shown that our proposed algorithm does not violate fairness constraints with high probability and with sublinear regret in the finite-horizon episodic setting. Through experiments in RiverSwim and MuJoCo environments, we demonstrate that our approach better ensures group fairness across multiple tasks compared to previous methods that lack explicit multi-task fairness constraints in both the finite-horizon setting and the infinite-horizon setting. Our results show that the proposed algorithm achieves smaller fairness gaps while maintaining comparable returns across different demographic groups and tasks, suggesting its potential for addressing fairness concerns in real-world multi-task RL applications.

Related papers

• Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning [70.96345405979179]
  The purpose of offline multi-task reinforcement learning (MTRL) is to develop a unified policy applicable to diverse tasks without the need for online environmental interaction. variations in task content and complexity pose significant challenges in policy formulation. We introduce the Harmony Multi-Task Decision Transformer (HarmoDT), a novel solution designed to identify an optimal harmony subspace of parameters for each task.
  arXiv  Detail & Related papers  (2024-11-02T05:49:14Z)
• Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks [53.44714413181162]
  This paper shows that when an agent is trained on a sufficiently diverse set of tasks, a generic policy-sharing algorithm with myopic exploration design can be sample-efficient. To the best of our knowledge, this is the first theoretical demonstration of the "exploration benefits" of MTRL.
  arXiv  Detail & Related papers  (2024-03-03T22:57:44Z)
• Towards Principled Task Grouping for Multi-Task Learning [14.3385939018772]
  We present a novel approach to task grouping in Multitask Learning (MTL) Our approach offers a more theoretically grounded method that does not rely on restrictive assumptions for constructing transfer gains.
  arXiv  Detail & Related papers  (2024-02-23T13:51:20Z)
• Offline Multi-task Transfer RL with Representational Penalization [26.114893629771736]
  We study the problem of representation transfer in offline Reinforcement Learning (RL) We propose an algorithm to compute pointwise uncertainty measures for the learnt representation.
  arXiv  Detail & Related papers  (2024-02-19T21:52:44Z)
• Provable Benefits of Multi-task RL under Non-Markovian Decision Making Processes [56.714690083118406]
  In multi-task reinforcement learning (RL) under Markov decision processes (MDPs), the presence of shared latent structures has been shown to yield significant benefits to the sample efficiency compared to single-task RL. We investigate whether such a benefit can extend to more general sequential decision making problems, such as partially observable MDPs (POMDPs) and more general predictive state representations (PSRs) We propose a provably efficient algorithm UMT-PSR for finding near-optimal policies for all PSRs, and demonstrate that the advantage of multi-task learning manifests if the joint model class of PSR
  arXiv  Detail & Related papers  (2023-10-20T14:50:28Z)
• Understanding the Complexity Gains of Single-Task RL with a Curriculum [83.46923851724408]
  Reinforcement learning (RL) problems can be challenging without well-shaped rewards. We provide a theoretical framework that reformulates a single-task RL problem as a multi-task RL problem defined by a curriculum. We show that sequentially solving each task in the multi-task RL problem is more computationally efficient than solving the original single-task problem.
  arXiv  Detail & Related papers  (2022-12-24T19:46:47Z)
• Reinforcement Learning with Stepwise Fairness Constraints [50.538878453547966]
  We introduce the study of reinforcement learning with stepwise fairness constraints. We provide learning algorithms with strong theoretical guarantees in regard to policy optimality and fairness violation.
  arXiv  Detail & Related papers  (2022-11-08T04:06:23Z)
• Variational Multi-Task Learning with Gumbel-Softmax Priors [105.22406384964144]
  Multi-task learning aims to explore task relatedness to improve individual tasks. We propose variational multi-task learning (VMTL), a general probabilistic inference framework for learning multiple related tasks.
  arXiv  Detail & Related papers  (2021-11-09T18:49:45Z)
• Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning [18.666340309506605]
  We are concerned with how group fairness as an ML fairness concept plays out in the multi-task scenario. In multi-task learning, several tasks are learned jointly to exploit task correlations for a more efficient inductive transfer. We propose a Multi-Task-Aware Fairness (MTA-F) approach to improve fairness in multi-task learning.
  arXiv  Detail & Related papers  (2021-06-04T20:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.