Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

• URL: http://arxiv.org/abs/2508.06336v1
• Date: Fri, 08 Aug 2025 14:11:15 GMT
• Title: Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
• Abstract summary: Unsupervised Partner Design (UPD) is a population-free, multi-agent reinforcement learning framework for robust ad-hoc teamwork.<n>UPD constructs diverse partners by mixing an ego agent's policy with random parameters and scores them using a variance-based learnability metric.<n>We show that UPD can be integrated with unsupervised environment design, resulting in the first method enabling fully unsupervised curricula.
• Score: 7.578942855148756
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce Unsupervised Partner Design (UPD) - a population-free, multi-agent reinforcement learning framework for robust ad-hoc teamwork that adaptively generates training partners without requiring pretrained partners or manual parameter tuning. UPD constructs diverse partners by stochastically mixing an ego agent's policy with biased random behaviours and scores them using a variance-based learnability metric that prioritises partners near the ego agent's current learning frontier. We show that UPD can be integrated with unsupervised environment design, resulting in the first method enabling fully unsupervised curricula over both level and partner distributions in a cooperative setting. Through extensive evaluations on Overcooked-AI and the Overcooked Generalisation Challenge, we demonstrate that this dynamic partner curriculum is highly effective: UPD consistently outperforms both population-based and population-free baselines as well as ablations. In a user study, we further show that UPD achieves higher returns than all baselines and was perceived as significantly more adaptive, more human-like, a better collaborator, and less frustrating.

Related papers

• Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration [1.4255659581428333]
  Multi-agent collaboration, especially in human-AI teaming, requires agents that can adapt to novel partners with diverse and dynamic behaviors.<n>We introduce Partner-Aware Skill Discovery (PASD), a DHRL framework that learns skills conditioned on partner behavior.<n>PASD introduces a contrastive intrinsic reward to capture patterns emerging from partner interactions, aligning skill representations across similar partners while maintaining discriminability across diverse strategies.
  arXiv  Detail & Related papers  (2026-05-23T02:26:46Z)
• Adaptive Human-AI Coordination via Hierarchical Action Disentanglement [1.4255659581428333]
  Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners.<n>We propose Intrinsic Action Disentanglement (IAD), a deep hierarchical reinforcement learning framework that learns distinct, partner-aware low-level action sequences conditioned on high-level latent skills.<n>We evaluate IAD in the Overcooked-AI domain across multiple layouts and diverse partner settings, including unseen simulated partners, a human-proxy model trained on human-human gameplay, and real human partners.
  arXiv  Detail & Related papers  (2026-05-23T01:59:41Z)
• Beyond Partner Diversity: An Influence-Based Team Steering Framework for Zero-Shot Human-Machine Teaming [1.6672631924542791]
  Influence-Based Team Steering is a framework that uses influence shaping to incentivize agents to discover diverse, high-performing team interaction patterns.<n>We assess IBTS on Overcooked-AI in both two-agent and three-agent settings, allowing us to test whether learned coordination structure transfers beyond dyadic interaction.<n>Our evaluation includes simulated partners, synthetic partner-style variation, and, to our knowledge, the first 30-subject Overcooked-AI HMT study involving two real human teammates and one machine teammate.
  arXiv  Detail & Related papers  (2026-05-14T20:34:16Z)
• Heterogeneous Agent Collaborative Reinforcement Learning [52.99813668995983]
  Heterogeneous Agent Collaborative Reinforcement Learning (HACRL)<n>Building on this paradigm, we propose HACPO, a collaborative RL algorithm that enables principled rollout sharing to maximize sample utilization and cross-agent knowledge transfer.<n>Experiments across diverse heterogeneous model combinations and reasoning benchmarks show that HACPO consistently improves all participating agents, outperforming GSPO by an average of 3.3% while using only half the rollout cost.
  arXiv  Detail & Related papers  (2026-03-03T05:09:49Z)
• Nested Training for Mutual Adaptation in Human-AI Teaming [30.247046563601202]
  Existing approaches aim to improve diversity in training partners to approximate human behavior, but these partners are static and fail to capture adaptive behavior of humans.<n>We model the human-robot teaming scenario as an Interactive Partially Observable Markov Decision Process (I-POMDP), explicitly modeling human adaptation as part of the state.<n>We train our method in a multi-episode, required cooperation setup in the Overcooked domain, comparing it against several baseline agents designed for human-robot teaming.
  arXiv  Detail & Related papers  (2026-02-18T23:07:48Z)
• Efficient Reinforcement Learning for Zero-Shot Coordination in Evolving Games [30.01934395713042]
  Zero-shot coordination is a key challenge in multi-agent game theory.<n>Population-based training has been proven to provide good zero-shot coordination performance.
  arXiv  Detail & Related papers  (2025-11-14T08:59:22Z)
• Learning "Partner-Aware" Collaborators in Multi-Party Collaboration [12.287537011305497]
  Large Language Models (LLMs) are increasingly bring deployed in agentic settings where they act as collaborators with humans.<n>This paper builds on the AI alignment and safe interruptability literature to offer novel theoretical insights on collaborative behavior.<n>We propose Interruptible Collaborative Roleplayer (ICR)-a novel partner-aware learning algorithm to train CG-optimal collaborators.
  arXiv  Detail & Related papers  (2025-10-26T00:05:48Z)
• CooT: Learning to Coordinate In-Context with Coordination Transformers [4.205946699819021]
  Coordination Transformers (CooT) is a novel in-context coordination framework that uses recent interaction histories to adapt to unseen partners rapidly.<n>Trained on interaction trajectories collected from diverse pairs of agents, CooT quickly learns effective coordination strategies without explicit supervision or fine-tuning.<n>Human evaluations confirm CooT as the most effective collaborative partner, while extensive ablations highlight its robustness, flexibility, and sensitivity to context in multi-agent scenarios.
  arXiv  Detail & Related papers  (2025-06-30T06:45:39Z)
• Partner Modelling Emerges in Recurrent Agents (But Only When It Matters) [4.845103288370202]
  We train simple model-free RNN agents to collaborate with a population of diverse partners.<n>We find structured partner modelling emerges when agents can influence partner behaviour by controlling task allocation.<n>Our results show that partner modelling can arise spontaneously in model-free agents -- but only under environmental conditions that impose the right kind of social pressure.
  arXiv  Detail & Related papers  (2025-05-22T22:24:12Z)
• The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design [15.896892078446989]
  Overcooked Generalisation Challenge (OGC) is a new benchmark for evaluating reinforcement learning (RL) agents on their ability to cooperate with unknown partners in unfamiliar environments.<n>The OGC extends Overcooked-AI to support dual curriculum design (DCD)<n>We evaluate state-of-the-art DCD algorithms alongside scalable neural architectures and find that current methods fail to produce agents that generalise effectively to novel layouts and unfamiliar partners.
  arXiv  Detail & Related papers  (2024-06-25T21:51:43Z)
• REVECA: Adaptive Planning and Trajectory-based Validation in Cooperative Language Agents using Information Relevance and Relative Proximity [5.365719315040012]
  REVECA is a novel cognitive architecture powered by GPT-4o-mini.<n>It enables efficient memory management, optimal planning, and cost-effective prevention of false planning.
  arXiv  Detail & Related papers  (2024-05-27T01:47:14Z)
• Large Language Model-based Human-Agent Collaboration for Complex Task Solving [94.3914058341565]
  We introduce the problem of Large Language Models (LLMs)-based human-agent collaboration for complex task-solving. We propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process.
  arXiv  Detail & Related papers  (2024-02-20T11:03:36Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• A Hierarchical Approach to Population Training for Human-AI Collaboration [20.860808795671343]
  We introduce a Hierarchical Reinforcement Learning (HRL) based method for Human-AI Collaboration. We demonstrate that our method is able to dynamically adapt to novel partners of different play styles and skill levels in the 2-player collaborative Overcooked game environment.
  arXiv  Detail & Related papers  (2023-05-26T07:53:12Z)
• PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination [52.991211077362586]
  We propose a policy ensemble method to increase the diversity of partners in the population. We then develop a context-aware method enabling the ego agent to analyze and identify the partner's potential policy primitives. In this way, the ego agent is able to learn more universal cooperative behaviors for collaborating with diverse partners.
  arXiv  Detail & Related papers  (2023-01-16T12:14:58Z)
• Conditional Imitation Learning for Multi-Agent Games [89.897635970366]
  We study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time. We propose a novel approach to address the difficulties of scalability and data scarcity. Our model learns a low-rank subspace over ego and partner agent strategies, then infers and adapts to a new partner strategy by interpolating in the subspace.
  arXiv  Detail & Related papers  (2022-01-05T04:40:13Z)
• Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous [66.6895109554163]
  Underlying the human ability to align goals with other agents is their ability to predict the intentions of others and actively update their own plans. We propose hierarchical predictive planning (HPP), a model-based reinforcement learning method for decentralized multiagent rendezvous.
  arXiv  Detail & Related papers  (2020-03-15T19:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.