Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning
- URL: http://arxiv.org/abs/2006.10412v4
- Date: Wed, 9 Jun 2021 16:23:52 GMT
- Title: Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning
- Authors: Arrasy Rahman, Niklas H\"opner, Filippos Christianos, Stefano V.
Albrecht
- Abstract summary: We build on graph neural networks to learn agent models and joint-action value models under varying team compositions.
We empirically demonstrate that our approach successfully models the effects other agents have on the learner, leading to policies that robustly adapt to dynamic team compositions.
- Score: 11.480994804659908
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Ad hoc teamwork is the challenging problem of designing an autonomous agent
which can adapt quickly to collaborate with teammates without prior
coordination mechanisms, including joint training. Prior work in this area has
focused on closed teams in which the number of agents is fixed. In this work,
we consider open teams by allowing agents with different fixed policies to
enter and leave the environment without prior notification. Our solution builds
on graph neural networks to learn agent models and joint-action value models
under varying team compositions. We contribute a novel action-value computation
that integrates the agent model and joint-action value model to produce
action-value estimates. We empirically demonstrate that our approach
successfully models the effects other agents have on the learner, leading to
policies that robustly adapt to dynamic team compositions and significantly
outperform several alternative methods.
Related papers
- Communication Learning in Multi-Agent Systems from Graph Modeling Perspective [62.13508281188895]
We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph.
We introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time.
arXiv Detail & Related papers (2024-11-01T05:56:51Z) - Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph.
Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
arXiv Detail & Related papers (2024-05-14T12:40:25Z) - Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server.
We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
arXiv Detail & Related papers (2024-03-11T09:21:11Z) - ProAgent: Building Proactive Cooperative Agents with Large Language
Models [89.53040828210945]
ProAgent is a novel framework that harnesses large language models to create proactive agents.
ProAgent can analyze the present state, and infer the intentions of teammates from observations.
ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
arXiv Detail & Related papers (2023-08-22T10:36:56Z) - Knowledge-based Reasoning and Learning under Partial Observability in Ad
Hoc Teamwork [4.454557728745761]
This paper introduces an architecture that determines an ad hoc agent's behavior based on non-monotonic logical reasoning.
It supports online selection, adaptation, and learning of the models that predict the other agents' behavior.
We show that the performance of our architecture is comparable or better than state of the art data-driven baselines in both simple and complex scenarios.
arXiv Detail & Related papers (2023-06-01T15:21:27Z) - Decentralized Adversarial Training over Graphs [55.28669771020857]
The vulnerability of machine learning models to adversarial attacks has been attracting considerable attention in recent years.
This work studies adversarial training over graphs, where individual agents are subjected to varied strength perturbation space.
arXiv Detail & Related papers (2023-03-23T15:05:16Z) - A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based
Policy Learning [11.998708550268978]
We develop a class of solutions for open ad hoc teamwork under full and partial observability.
We show that our solution can learn efficient policies in open ad hoc teamwork in fully and partially observable cases.
arXiv Detail & Related papers (2022-10-11T13:44:44Z) - Toward a Reasoning and Learning Architecture for Ad Hoc Teamwork [4.454557728745761]
We present an architecture for ad hoc teamwork, which refers to collaboration in a team of agents without prior coordination.
Our architecture combines the principles of knowledge-based and data-driven reasoning and learning.
We use the benchmark simulated multiagent collaboration domain Fort Attack to demonstrate that our architecture supports adaptation to unforeseen changes.
arXiv Detail & Related papers (2022-08-24T13:57:33Z) - Conditional Imitation Learning for Multi-Agent Games [89.897635970366]
We study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time.
We propose a novel approach to address the difficulties of scalability and data scarcity.
Our model learns a low-rank subspace over ego and partner agent strategies, then infers and adapts to a new partner strategy by interpolating in the subspace.
arXiv Detail & Related papers (2022-01-05T04:40:13Z) - Learning to Model Opponent Learning [11.61673411387596]
Multi-Agent Reinforcement Learning (MARL) considers settings in which a set of coexisting agents interact with one another and their environment.
This poses a great challenge for value function-based algorithms whose convergence usually relies on the assumption of a stationary environment.
We develop a novel approach to modelling an opponent's learning dynamics which we term Learning to Model Opponent Learning (LeMOL)
arXiv Detail & Related papers (2020-06-06T17:19:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.