Multi Type Mean Field Reinforcement Learning
- URL: http://arxiv.org/abs/2002.02513v7
- Date: Tue, 21 Jun 2022 08:40:51 GMT
- Title: Multi Type Mean Field Reinforcement Learning
- Authors: Sriram Ganapathi Subramanian and Pascal Poupart and Matthew E. Taylor
and Nidhi Hegde
- Abstract summary: We extend mean field multiagent algorithms to multiple types.
We conduct experiments on three different testbeds for the field of many agent reinforcement learning.
- Score: 26.110052366068533
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Mean field theory provides an effective way of scaling multiagent
reinforcement learning algorithms to environments with many agents that can be
abstracted by a virtual mean agent. In this paper, we extend mean field
multiagent algorithms to multiple types. The types enable the relaxation of a
core assumption in mean field reinforcement learning, which is that all agents
in the environment are playing almost similar strategies and have the same
goal. We conduct experiments on three different testbeds for the field of many
agent reinforcement learning, based on the standard MAgents framework. We
consider two different kinds of mean field environments: a) Games where agents
belong to predefined types that are known a priori and b) Games where the type
of each agent is unknown and therefore must be learned based on observations.
We introduce new algorithms for each type of game and demonstrate their
superior performance over state of the art algorithms that assume that all
agents belong to the same type and other baseline algorithms in the MAgent
framework.
Related papers
- AgentGym: Evolving Large Language Model-based Agents across Diverse Environments [116.97648507802926]
Large language models (LLMs) are considered a promising foundation to build such agents.
We take the first step towards building generally-capable LLM-based agents with self-evolution ability.
We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration.
arXiv Detail & Related papers (2024-06-06T15:15:41Z) - Generating Personas for Games with Multimodal Adversarial Imitation
Learning [47.70823327747952]
Reinforcement learning has been widely successful in producing agents capable of playing games at a human level.
Going beyond reinforcement learning is necessary to model a wide range of human playstyles.
This paper presents a novel imitation learning approach to generate multiple persona policies for playtesting.
arXiv Detail & Related papers (2023-08-15T06:58:19Z) - MADiff: Offline Multi-agent Learning with Diffusion Models [79.18130544233794]
Diffusion model (DM) recently achieved huge success in various scenarios including offline reinforcement learning.
We propose MADiff, a novel generative multi-agent learning framework to tackle this problem.
Our experiments show the superior performance of MADiff compared to baseline algorithms in a wide range of multi-agent learning tasks.
arXiv Detail & Related papers (2023-05-27T02:14:09Z) - Multi-agent Deep Covering Skill Discovery [50.812414209206054]
We propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space.
Also, we propose a novel framework to adopt the multi-agent options in the MARL process.
We show that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options.
arXiv Detail & Related papers (2022-10-07T00:40:59Z) - MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent
Reinforcement Learning [38.77840067555711]
We propose the first set of interpretable MARL algorithms that extract decision-tree policies from neural networks trained with MARL.
The first algorithm, IVIPER, extends VIPER, a recent method for single-agent interpretable RL, to the multi-agent setting.
To better capture coordination between agents, we propose a novel centralized decision-tree training algorithm, MAVIPER.
arXiv Detail & Related papers (2022-05-25T02:38:10Z) - Learning Meta Representations for Agents in Multi-Agent Reinforcement
Learning [12.170248966278281]
In multi-agent reinforcement learning, behaviors that agents learn in a single Markov Game (MG) are typically confined to the given agent number.
In this work, our focus is on creating agents that can generalize across population-varying MGs.
Instead of learning a unimodal policy, each agent learns a policy set comprising effective strategies across a variety of games.
arXiv Detail & Related papers (2021-08-30T04:30:53Z) - Model Free Reinforcement Learning Algorithm for Stationary Mean field
Equilibrium for Multiple Types of Agents [43.21120427632336]
We consider a multi-agent strategic interaction over an infinite horizon where agents can be of multiple types.
Each agent has a private state; the state evolves depending on the distribution of the state of the agents of different types and the action of the agent.
We show how such kind of interaction can model the cyber attacks among defenders and adversaries.
arXiv Detail & Related papers (2020-12-31T00:12:46Z) - A Policy Gradient Algorithm for Learning to Learn in Multiagent
Reinforcement Learning [47.154539984501895]
We propose a novel meta-multiagent policy gradient theorem that accounts for the non-stationary policy dynamics inherent to multiagent learning settings.
This is achieved by modeling our gradient updates to consider both an agent's own non-stationary policy dynamics and the non-stationary policy dynamics of other agents in the environment.
arXiv Detail & Related papers (2020-10-31T22:50:21Z) - Non-cooperative Multi-agent Systems with Exploring Agents [10.736626320566707]
We develop a prescriptive model of multi-agent behavior using Markov games.
We focus on models in which the agents play "exploration but near optimum strategies"
arXiv Detail & Related papers (2020-05-25T19:34:29Z) - Never Give Up: Learning Directed Exploration Strategies [63.19616370038824]
We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies.
We construct an episodic memory-based intrinsic reward using k-nearest neighbors over the agent's recent experience to train the directed exploratory policies.
A self-supervised inverse dynamics model is used to train the embeddings of the nearest neighbour lookup, biasing the novelty signal towards what the agent can control.
arXiv Detail & Related papers (2020-02-14T13:57:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.