Towards Learning Multi-agent Negotiations via Self-Play
- URL: http://arxiv.org/abs/2001.10208v1
- Date: Tue, 28 Jan 2020 08:37:33 GMT
- Title: Towards Learning Multi-agent Negotiations via Self-Play
- Authors: Yichuan Charlie Tang
- Abstract summary: We show how an iterative procedure of self-play can create progressively more diverse environments.
This leads to the learning of sophisticated and robust multi-agent policies.
We show a dramatic improvement in the success rate of merging maneuvers from 63% to over 98%.
- Score: 2.28438857884398
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Making sophisticated, robust, and safe sequential decisions is at the heart
of intelligent systems. This is especially critical for planning in complex
multi-agent environments, where agents need to anticipate other agents'
intentions and possible future actions. Traditional methods formulate the
problem as a Markov Decision Process, but the solutions often rely on various
assumptions and become brittle when presented with corner cases. In contrast,
deep reinforcement learning (Deep RL) has been very effective at finding
policies by simultaneously exploring, interacting, and learning from
environments. Leveraging the powerful Deep RL paradigm, we demonstrate that an
iterative procedure of self-play can create progressively more diverse
environments, leading to the learning of sophisticated and robust multi-agent
policies. We demonstrate this in a challenging multi-agent simulation of
merging traffic, where agents must interact and negotiate with others in order
to successfully merge on or off the road. While the environment starts off
simple, we increase its complexity by iteratively adding an increasingly
diverse set of agents to the agent "zoo" as training progresses. Qualitatively,
we find that through self-play, our policies automatically learn interesting
behaviors such as defensive driving, overtaking, yielding, and the use of
signal lights to communicate intentions to other agents. In addition,
quantitatively, we show a dramatic improvement of the success rate of merging
maneuvers from 63% to over 98%.
Related papers
- Active Legibility in Multiagent Reinforcement Learning [3.7828554251478734]
The legibility-oriented framework allows agents to conduct legible actions so as to help others optimise their behaviors.
The experimental results demonstrate that the new framework is more efficient and costs less training time compared to several multiagent reinforcement learning algorithms.
arXiv Detail & Related papers (2024-10-28T12:15:49Z) - Learning Emergence of Interaction Patterns across Independent RL Agents in Multi-Agent Environments [3.0284592792243794]
Bottom Up Network (BUN) treats the collective of multi-agents as a unified entity.
Our empirical evaluations across a variety of cooperative multi-agent scenarios, including tasks such as cooperative navigation and traffic control, consistently demonstrate BUN's superiority over baseline methods with substantially reduced computational costs.
arXiv Detail & Related papers (2024-10-03T14:25:02Z) - DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement
Learning [84.22561239481901]
We propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents.
We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement.
arXiv Detail & Related papers (2023-12-10T06:03:57Z) - Hierarchical Reinforcement Learning with Opponent Modeling for
Distributed Multi-agent Cooperation [13.670618752160594]
Deep reinforcement learning (DRL) provides a promising approach for multi-agent cooperation through the interaction of the agents and environments.
Traditional DRL solutions suffer from the high dimensions of multiple agents with continuous action space during policy search.
We propose a hierarchical reinforcement learning approach with high-level decision-making and low-level individual control for efficient policy search.
arXiv Detail & Related papers (2022-06-25T19:09:29Z) - Q-Mixing Network for Multi-Agent Pathfinding in Partially Observable
Grid Environments [62.997667081978825]
We consider the problem of multi-agent navigation in partially observable grid environments.
We suggest utilizing the reinforcement learning approach when the agents, first, learn the policies that map observations to actions and then follow these policies to reach their goals.
arXiv Detail & Related papers (2021-08-13T09:44:47Z) - Learning Latent Representations to Influence Multi-Agent Interaction [65.44092264843538]
We propose a reinforcement learning-based framework for learning latent representations of an agent's policy.
We show that our approach outperforms the alternatives and learns to influence the other agent.
arXiv Detail & Related papers (2020-11-12T19:04:26Z) - A Policy Gradient Algorithm for Learning to Learn in Multiagent
Reinforcement Learning [47.154539984501895]
We propose a novel meta-multiagent policy gradient theorem that accounts for the non-stationary policy dynamics inherent to multiagent learning settings.
This is achieved by modeling our gradient updates to consider both an agent's own non-stationary policy dynamics and the non-stationary policy dynamics of other agents in the environment.
arXiv Detail & Related papers (2020-10-31T22:50:21Z) - SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for
Autonomous Driving [96.50297622371457]
Multi-agent interaction is a fundamental aspect of autonomous driving in the real world.
Despite more than a decade of research and development, the problem of how to interact with diverse road users in diverse scenarios remains largely unsolved.
We develop a dedicated simulation platform called SMARTS that generates diverse and competent driving interactions.
arXiv Detail & Related papers (2020-10-19T18:26:10Z) - Learning to Incentivize Other Learning Agents [73.03133692589532]
We show how to equip RL agents with the ability to give rewards directly to other agents, using a learned incentive function.
Such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games.
Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.
arXiv Detail & Related papers (2020-06-10T20:12:38Z) - Individual specialization in multi-task environments with multiagent
reinforcement learners [0.0]
There is a growing interest in Multi-Agent Reinforcement Learning (MARL) as the first steps towards building general intelligent agents.
Previous results point us towards increased conditions for coordination, efficiency/fairness, and common-pool resource sharing.
We further study coordination in multi-task environments where several rewarding tasks can be performed and thus agents don't necessarily need to perform well in all tasks, but under certain conditions may specialize.
arXiv Detail & Related papers (2019-12-29T15:20:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.