A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning

• URL: http://arxiv.org/abs/2210.05448v2
• Date: Sat, 28 Oct 2023 18:29:29 GMT
• Title: A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning
• Authors: Arrasy Rahman and Ignacio Carlucho and Niklas H\"opner and Stefano V. Albrecht
• Abstract summary: We develop a class of solutions for open ad hoc teamwork under full and partial observability. We show that our solution can learn efficient policies in open ad hoc teamwork in fully and partially observable cases.
• Score: 11.998708550268978
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open ad hoc teamwork is the problem of training a single agent to efficiently collaborate with an unknown group of teammates whose composition may change over time. A variable team composition creates challenges for the agent, such as the requirement to adapt to new team dynamics and dealing with changing state vector sizes. These challenges are aggravated in real-world applications in which the controlled agent only has a partial view of the environment. In this work, we develop a class of solutions for open ad hoc teamwork under full and partial observability. We start by developing a solution for the fully observable case that leverages graph neural network architectures to obtain an optimal policy based on reinforcement learning. We then extend this solution to partially observable scenarios by proposing different methodologies that maintain belief estimates over the latent environment states and team composition. These belief estimates are combined with our solution for the fully observable case to compute an agent's optimal policy under partial observability in open ad hoc teamwork. Empirical results demonstrate that our solution can learn efficient policies in open ad hoc teamwork in fully and partially observable cases. Further analysis demonstrates that our methods' success is a result of effectively learning the effects of teammates' actions while also inferring the inherent state of the environment under partial observability.

Related papers

• N-Agent Ad Hoc Teamwork [36.10108537776956]
  Current approaches to learning cooperative multi-agent behaviors assume relatively restrictive settings. This paper formalizes the problem, and proposes the Policy Optimization with Agent Modelling (POAM) algorithm. POAM is a policy gradient, multi-agent reinforcement learning approach to the NAHT problem, that enables adaptation to diverse teammate behaviors.
  arXiv  Detail & Related papers  (2024-04-16T17:13:08Z)
• Open Ad Hoc Teamwork with Cooperative Game Theory [28.605478081031215]
  Ad hoc teamwork poses a challenging problem, requiring the design of an agent to collaborate with teammates without prior coordination or joint training. One promising solution is leveraging the generalizability of graph neural networks to handle an unrestricted number of agents. We propose a novel algorithm named CIAO, based on the game's framework, with additional provable implementation tricks that can facilitate learning.
  arXiv  Detail & Related papers  (2024-02-23T11:04:33Z)
• Making Friends in the Dark: Ad Hoc Teamwork Under Partial Observability [11.786470737937638]
  This paper introduces a formal definition of the setting of ad hoc teamwork under partial observability. Our results in 70 POMDPs from 11 domains show that our approach is not only effective in assisting unknown teammates in solving unknown tasks but is also robust in scaling to more challenging problems.
  arXiv  Detail & Related papers  (2023-09-30T16:40:50Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Knowledge-based Reasoning and Learning under Partial Observability in Ad Hoc Teamwork [4.454557728745761]
  This paper introduces an architecture that determines an ad hoc agent's behavior based on non-monotonic logical reasoning. It supports online selection, adaptation, and learning of the models that predict the other agents' behavior. We show that the performance of our architecture is comparable or better than state of the art data-driven baselines in both simple and complex scenarios.
  arXiv  Detail & Related papers  (2023-06-01T15:21:27Z)
• A Reinforcement Learning-assisted Genetic Programming Algorithm for Team Formation Problem Considering Person-Job Matching [70.28786574064694]
  A reinforcement learning-assisted genetic programming algorithm (RL-GP) is proposed to enhance the quality of solutions. The hyper-heuristic rules obtained through efficient learning can be utilized as decision-making aids when forming project teams.
  arXiv  Detail & Related papers  (2023-04-08T14:32:12Z)
• Detecting and Optimising Team Interactions in Software Development [58.720142291102135]
  This paper presents a data-driven approach to detect the functional interaction structure for software development teams. Our approach considers differences in the activity levels of team members and uses a block-constrained configuration model. We show how our approach enables teams to compare their functional interaction structure against synthetically created benchmark scenarios.
  arXiv  Detail & Related papers  (2023-02-28T14:53:29Z)
• Conditional Imitation Learning for Multi-Agent Games [89.897635970366]
  We study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time. We propose a novel approach to address the difficulties of scalability and data scarcity. Our model learns a low-rank subspace over ego and partner agent strategies, then infers and adapts to a new partner strategy by interpolating in the subspace.
  arXiv  Detail & Related papers  (2022-01-05T04:40:13Z)
• Distributed Adaptive Learning Under Communication Constraints [54.22472738551687]
  This work examines adaptive distributed learning strategies designed to operate under communication constraints. We consider a network of agents that must solve an online optimization problem from continual observation of streaming data.
  arXiv  Detail & Related papers  (2021-12-03T19:23:48Z)
• Q-Mixing Network for Multi-Agent Pathfinding in Partially Observable Grid Environments [62.997667081978825]
  We consider the problem of multi-agent navigation in partially observable grid environments. We suggest utilizing the reinforcement learning approach when the agents, first, learn the policies that map observations to actions and then follow these policies to reach their goals.
  arXiv  Detail & Related papers  (2021-08-13T09:44:47Z)
• Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning [11.480994804659908]
  We build on graph neural networks to learn agent models and joint-action value models under varying team compositions. We empirically demonstrate that our approach successfully models the effects other agents have on the learner, leading to policies that robustly adapt to dynamic team compositions.
  arXiv  Detail & Related papers  (2020-06-18T10:39:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.