Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration

• URL: http://arxiv.org/abs/2605.24352v1
• Date: Sat, 23 May 2026 02:26:46 GMT
• Title: Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration
• Abstract summary: Multi-agent collaboration, especially in human-AI teaming, requires agents that can adapt to novel partners with diverse and dynamic behaviors.<n>We introduce Partner-Aware Skill Discovery (PASD), a DHRL framework that learns skills conditioned on partner behavior.<n>PASD introduces a contrastive intrinsic reward to capture patterns emerging from partner interactions, aligning skill representations across similar partners while maintaining discriminability across diverse strategies.
• Score: 1.4255659581428333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent collaboration, especially in human-AI teaming, requires agents that can adapt to novel partners with diverse and dynamic behaviors. Conventional Deep Hierarchical Reinforcement Learning (DHRL) methods focus on agent-centric rewards and overlook partner behavior, leading to shortcut learning, where skills exploit spurious information instead of adapting to partners' dynamic behaviors. This limitation undermines agents' ability to adapt and coordinate effectively with novel partners. We introduce Partner-Aware Skill Discovery (PASD), a DHRL framework that learns skills conditioned on partner behavior. PASD introduces a contrastive intrinsic reward to capture patterns emerging from partner interactions, aligning skill representations across similar partners while maintaining discriminability across diverse strategies. By structuring the skill space based on partner interactions, this approach mitigates shortcut learning and promotes behavioral consistency, enabling robust and adaptive coordination. We extensively evaluate PASD in the Overcooked-AI benchmark with a diverse population of partners characterized by varying skill levels and play styles. We further evaluate the approach with human proxy models trained from human-human gameplay trajectories. PASD consistently outperforms existing population-based and hierarchical baselines, demonstrating transferable skill learning that generalizes across a wide range of partner behaviors. Analysis of learned skill representations shows that PASD adapts effectively to diverse partner behaviors, highlighting its robustness in human-AI collaboration.

Related papers

• HRIBench: Benchmarking Interaction-Centric Human-Robot Collaboration [62.19372189311434]
  We introduce HRIBench, a diagnostic benchmark for intent-aware human-robot collaboration based on executable scenarios.<n>HRIBench represents collaborative tasks as structured scenario scripts that explicitly model agent roles, temporal dependencies, coordination constraints, and human behavior distributions.<n>Results show that current foundation robot policies struggle substantially in collaborative settings despite strong manipulation ability.
  arXiv  Detail & Related papers  (2026-07-05T09:15:35Z)
• Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration [54.18136393879849]
  We present ALMANAC, a dataset of action-level mental model ANnotations for Agent Collaboration.<n>ALMANAC contains 2,987 collaboration actions, each paired with theory-informed mental model annotations.<n>Our results demonstrate ALMANAC's utility in evaluating models' ability to simulate human collaborative behaviors.
  arXiv  Detail & Related papers  (2026-06-04T16:56:12Z)
• Adaptive Human-AI Coordination via Hierarchical Action Disentanglement [1.4255659581428333]
  Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners.<n>We propose Intrinsic Action Disentanglement (IAD), a deep hierarchical reinforcement learning framework that learns distinct, partner-aware low-level action sequences conditioned on high-level latent skills.<n>We evaluate IAD in the Overcooked-AI domain across multiple layouts and diverse partner settings, including unseen simulated partners, a human-proxy model trained on human-human gameplay, and real human partners.
  arXiv  Detail & Related papers  (2026-05-23T01:59:41Z)
• Adaptively Coordinating with Novel Partners via Learned Latent Strategies [19.014669675808133]
  We introduce a strategy-conditioned cooperator framework that learns to represent, categorize, and adapt to a broad range of potential partner strategies in real-time.<n>Our approach encodes strategies with a variational autoencoder to learn a latent strategy space from agent trajectory data.<n>We leverage a fixed-share regret minimization algorithm that dynamically infers and adjusts the partner's strategy estimation during interaction.
  arXiv  Detail & Related papers  (2025-11-16T19:45:35Z)
• Learning "Partner-Aware" Collaborators in Multi-Party Collaboration [12.287537011305497]
  Large Language Models (LLMs) are increasingly bring deployed in agentic settings where they act as collaborators with humans.<n>This paper builds on the AI alignment and safe interruptability literature to offer novel theoretical insights on collaborative behavior.<n>We propose Interruptible Collaborative Roleplayer (ICR)-a novel partner-aware learning algorithm to train CG-optimal collaborators.
  arXiv  Detail & Related papers  (2025-10-26T00:05:48Z)
• Unsupervised Partner Design Enables Robust Ad-hoc Teamwork [7.578942855148756]
  Unsupervised Partner Design (UPD) is a population-free, multi-agent reinforcement learning framework for robust ad-hoc teamwork.<n>UPD constructs diverse partners by mixing an ego agent's policy with random parameters and scores them using a variance-based learnability metric.<n>We show that UPD can be integrated with unsupervised environment design, resulting in the first method enabling fully unsupervised curricula.
  arXiv  Detail & Related papers  (2025-08-08T14:11:15Z)
• CooT: Learning to Coordinate In-Context with Coordination Transformers [10.888155149916967]
  Coordination Transformers (coot) is a novel in-context coordination framework that rapidly adapts to unseen partners.<n>coot consistently outperforms baselines including population-based approaches, gradient-based fine-tuning, and a Meta-RL-inspired contextual adaptation method.<n>By contrast, coot achieves stable, rapid in-context adaptation and is consistently ranked the most effective collaborator in human evaluations.
  arXiv  Detail & Related papers  (2025-06-30T06:45:39Z)
• Adaptive Coordination in Social Embodied Rearrangement [49.35582108902819]
  We study zero-shot coordination (ZSC) in this task, where an agent collaborates with a new partner, emulating a scenario where a robot collaborates with a new human partner. We propose Behavior Diversity Play (BDP), a novel ZSC approach that encourages diversity through a discriminability objective. Our results demonstrate that BDP learns adaptive agents that can tackle visual coordination, and zero-shot generalize to new partners in unseen environments, achieving 35% higher success and 32% higher efficiency compared to baselines.
  arXiv  Detail & Related papers  (2023-05-31T18:05:51Z)
• A Hierarchical Approach to Population Training for Human-AI Collaboration [20.860808795671343]
  We introduce a Hierarchical Reinforcement Learning (HRL) based method for Human-AI Collaboration. We demonstrate that our method is able to dynamically adapt to novel partners of different play styles and skill levels in the 2-player collaborative Overcooked game environment.
  arXiv  Detail & Related papers  (2023-05-26T07:53:12Z)
• Learning to Influence Human Behavior with Offline Reinforcement Learning [70.7884839812069]
  We focus on influence in settings where there is a need to capture human suboptimality. Experiments online with humans is potentially unsafe, and creating a high-fidelity simulator of the environment is often impractical. We show that offline reinforcement learning can learn to effectively influence suboptimal humans by extending and combining elements of observed human-human behavior.
  arXiv  Detail & Related papers  (2023-03-03T23:41:55Z)
• PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination [52.991211077362586]
  We propose a policy ensemble method to increase the diversity of partners in the population. We then develop a context-aware method enabling the ego agent to analyze and identify the partner's potential policy primitives. In this way, the ego agent is able to learn more universal cooperative behaviors for collaborating with diverse partners.
  arXiv  Detail & Related papers  (2023-01-16T12:14:58Z)
• Conditional Imitation Learning for Multi-Agent Games [89.897635970366]
  We study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time. We propose a novel approach to address the difficulties of scalability and data scarcity. Our model learns a low-rank subspace over ego and partner agent strategies, then infers and adapts to a new partner strategy by interpolating in the subspace.
  arXiv  Detail & Related papers  (2022-01-05T04:40:13Z)
• On the Critical Role of Conventions in Adaptive Human-AI Collaboration [73.21967490610142]
  We propose a learning framework that teases apart rule-dependent representation from convention-dependent representation. We experimentally validate our approach on three collaborative tasks varying in complexity.
  arXiv  Detail & Related papers  (2021-04-07T02:46:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.