Survey on Fair Reinforcement Learning: Theory and Practice
- URL: http://arxiv.org/abs/2205.10032v1
- Date: Fri, 20 May 2022 09:07:28 GMT
- Title: Survey on Fair Reinforcement Learning: Theory and Practice
- Authors: Pratik Gajane, Akrati Saxena, Maryam Tavakol, George Fletcher, and
Mykola Pechenizkiy
- Abstract summary: We provide an extensive overview of fairness approaches that have been implemented via a reinforcement learning (RL) framework.
We discuss various practical applications in which RL methods have been applied to achieve a fair solution with high accuracy.
We highlight a few major issues to explore in order to advance the field of fair-RL.
- Score: 9.783469272270896
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Fairness-aware learning aims at satisfying various fairness constraints in
addition to the usual performance criteria via data-driven machine learning
techniques. Most of the research in fairness-aware learning employs the setting
of fair-supervised learning. However, many dynamic real-world applications can
be better modeled using sequential decision-making problems and fair
reinforcement learning provides a more suitable alternative for addressing
these problems. In this article, we provide an extensive overview of fairness
approaches that have been implemented via a reinforcement learning (RL)
framework. We discuss various practical applications in which RL methods have
been applied to achieve a fair solution with high accuracy. We further include
various facets of the theory of fair reinforcement learning, organizing them
into single-agent RL, multi-agent RL, long-term fairness via RL, and offline
learning. Moreover, we highlight a few major issues to explore in order to
advance the field of fair-RL, namely - i) correcting societal biases, ii)
feasibility of group fairness or individual fairness, and iii) explainability
in RL. Our work is beneficial for both researchers and practitioners as we
discuss articles providing mathematical guarantees as well as articles with
empirical studies on real-world problems.
Related papers
- Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review [50.67937325077047]
This paper is devoted to a comprehensive review of realizing the sample efficiency and generalization of RL algorithms through transfer and inverse reinforcement learning (T-IRL)
Our findings denote that a majority of recent research works have dealt with the aforementioned challenges by utilizing human-in-the-loop and sim-to-real strategies.
Under the IRL structure, training schemes that require a low number of experience transitions and extension of such frameworks to multi-agent and multi-intention problems have been the priority of researchers in recent years.
arXiv Detail & Related papers (2024-11-15T15:18:57Z) - Multi-Agent Reinforcement Learning from Human Feedback: Data Coverage and Algorithmic Techniques [65.55451717632317]
We study Multi-Agent Reinforcement Learning from Human Feedback (MARLHF), exploring both theoretical foundations and empirical validations.
We define the task as identifying Nash equilibrium from a preference-only offline dataset in general-sum games.
Our findings underscore the multifaceted approach required for MARLHF, paving the way for effective preference-based multi-agent systems.
arXiv Detail & Related papers (2024-09-01T13:14:41Z) - Fairness in Reinforcement Learning: A Survey [0.0]
We survey the literature to provide the most up-to-date snapshot of the frontiers of fairness in reinforcement learning.
We highlight the methodologies researchers used to implement fairness in single- and multi-agent RL systems.
We critically examine gaps in the literature, such as understanding fairness in the context of RLHF.
arXiv Detail & Related papers (2024-05-11T04:36:46Z) - Sample Efficient Myopic Exploration Through Multitask Reinforcement
Learning with Diverse Tasks [53.44714413181162]
This paper shows that when an agent is trained on a sufficiently diverse set of tasks, a generic policy-sharing algorithm with myopic exploration design can be sample-efficient.
To the best of our knowledge, this is the first theoretical demonstration of the "exploration benefits" of MTRL.
arXiv Detail & Related papers (2024-03-03T22:57:44Z) - Fair Off-Policy Learning from Observational Data [30.77874108094485]
We propose a novel framework for fair off-policy learning.
We first formalize different fairness notions for off-policy learning.
We then propose a neural network-based framework to learn optimal policies under different fairness notions.
arXiv Detail & Related papers (2023-03-15T10:47:48Z) - Reinforcement Learning with Stepwise Fairness Constraints [50.538878453547966]
We introduce the study of reinforcement learning with stepwise fairness constraints.
We provide learning algorithms with strong theoretical guarantees in regard to policy optimality and fairness violation.
arXiv Detail & Related papers (2022-11-08T04:06:23Z) - Reinforcement Learning Applied to Trading Systems: A Survey [5.118560450410779]
The recent achievements and the notoriety of Reinforcement Learning have increased its adoption in trading tasks.
This review attempts to promote the development of this field of study by researchers' commitment to standards adherence.
arXiv Detail & Related papers (2022-11-01T21:26:12Z) - Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning [92.18524491615548]
Contrastive self-supervised learning has been successfully integrated into the practice of (deep) reinforcement learning (RL)
We study how RL can be empowered by contrastive learning in a class of Markov decision processes (MDPs) and Markov games (MGs) with low-rank transitions.
Under the online setting, we propose novel upper confidence bound (UCB)-type algorithms that incorporate such a contrastive loss with online RL algorithms for MDPs or MGs.
arXiv Detail & Related papers (2022-07-29T17:29:08Z) - A Probabilistic Interpretation of Self-Paced Learning with Applications
to Reinforcement Learning [30.69129405392038]
We present an approach for automated curriculum generation in reinforcement learning.
We formalize the well-known self-paced learning paradigm as inducing a distribution over training tasks.
Experiments show that training on this induced distribution helps to avoid poor local optima across RL algorithms.
arXiv Detail & Related papers (2021-02-25T21:06:56Z) - Towards Continual Reinforcement Learning: A Review and Perspectives [69.48324517535549]
We aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL)
While still in its early days, the study of continual RL has the promise to develop better incremental reinforcement learners.
These include applications such as those in the fields of healthcare, education, logistics, and robotics.
arXiv Detail & Related papers (2020-12-25T02:35:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.