Adaptive Human-AI Coordination via Hierarchical Action Disentanglement

• URL: http://arxiv.org/abs/2605.24343v1
• Date: Sat, 23 May 2026 01:59:41 GMT
• Title: Adaptive Human-AI Coordination via Hierarchical Action Disentanglement
• Abstract summary: Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners.<n>We propose Intrinsic Action Disentanglement (IAD), a deep hierarchical reinforcement learning framework that learns distinct, partner-aware low-level action sequences conditioned on high-level latent skills.<n>We evaluate IAD in the Overcooked-AI domain across multiple layouts and diverse partner settings, including unseen simulated partners, a human-proxy model trained on human-human gameplay, and real human partners.
• Score: 1.4255659581428333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners. Existing methods often collapse to a single dominant behavior or learn poorly aligned skills, limiting effective coordination. We propose Intrinsic Action Disentanglement (IAD), a deep hierarchical reinforcement learning (DHRL) framework that learns distinct, partner-aware low-level action sequences conditioned on high-level latent skills. IAD introduces an intrinsic reward that explicitly encourages disentangled action distributions of the agent's low-level policy across skills, yielding an interpretable mapping between high-level decisions and partner-specific behavioral responses. By capturing temporally extended interaction patterns, IAD enables flexible adaptation to heterogeneous partner dynamics under distributional shift. We evaluate IAD in the Overcooked-AI domain across multiple layouts and diverse partner settings, including unseen simulated partners, a human-proxy model trained on human-human gameplay, and real human partners. Results show that IAD consistently outperforms strong baselines and achieves more reliable, adaptive coordination across all settings.

Related papers

• HRIBench: Benchmarking Interaction-Centric Human-Robot Collaboration [62.19372189311434]
  We introduce HRIBench, a diagnostic benchmark for intent-aware human-robot collaboration based on executable scenarios.<n>HRIBench represents collaborative tasks as structured scenario scripts that explicitly model agent roles, temporal dependencies, coordination constraints, and human behavior distributions.<n>Results show that current foundation robot policies struggle substantially in collaborative settings despite strong manipulation ability.
  arXiv  Detail & Related papers  (2026-07-05T09:15:35Z)
• Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration [1.4255659581428333]
  Multi-agent collaboration, especially in human-AI teaming, requires agents that can adapt to novel partners with diverse and dynamic behaviors.<n>We introduce Partner-Aware Skill Discovery (PASD), a DHRL framework that learns skills conditioned on partner behavior.<n>PASD introduces a contrastive intrinsic reward to capture patterns emerging from partner interactions, aligning skill representations across similar partners while maintaining discriminability across diverse strategies.
  arXiv  Detail & Related papers  (2026-05-23T02:26:46Z)
• SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy [59.86199649493784]
  Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence.<n>We present SynAgent, a framework that enables scalable and physically plausible cooperative manipulation.<n>In experiments, SynAgent significantly outperforms existing baselines in both cooperative imitation and trajectory-conditioned control.
  arXiv  Detail & Related papers  (2026-04-20T17:46:20Z)
• Nested Training for Mutual Adaptation in Human-AI Teaming [30.247046563601202]
  Existing approaches aim to improve diversity in training partners to approximate human behavior, but these partners are static and fail to capture adaptive behavior of humans.<n>We model the human-robot teaming scenario as an Interactive Partially Observable Markov Decision Process (I-POMDP), explicitly modeling human adaptation as part of the state.<n>We train our method in a multi-episode, required cooperation setup in the Overcooked domain, comparing it against several baseline agents designed for human-robot teaming.
  arXiv  Detail & Related papers  (2026-02-18T23:07:48Z)
• Unsupervised Partner Design Enables Robust Ad-hoc Teamwork [7.578942855148756]
  Unsupervised Partner Design (UPD) is a population-free, multi-agent reinforcement learning framework for robust ad-hoc teamwork.<n>UPD constructs diverse partners by mixing an ego agent's policy with random parameters and scores them using a variance-based learnability metric.<n>We show that UPD can be integrated with unsupervised environment design, resulting in the first method enabling fully unsupervised curricula.
  arXiv  Detail & Related papers  (2025-08-08T14:11:15Z)
• CooT: Learning to Coordinate In-Context with Coordination Transformers [10.888155149916967]
  Coordination Transformers (coot) is a novel in-context coordination framework that rapidly adapts to unseen partners.<n>coot consistently outperforms baselines including population-based approaches, gradient-based fine-tuning, and a Meta-RL-inspired contextual adaptation method.<n>By contrast, coot achieves stable, rapid in-context adaptation and is consistently ranked the most effective collaborator in human evaluations.
  arXiv  Detail & Related papers  (2025-06-30T06:45:39Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• A Hierarchical Approach to Population Training for Human-AI Collaboration [20.860808795671343]
  We introduce a Hierarchical Reinforcement Learning (HRL) based method for Human-AI Collaboration. We demonstrate that our method is able to dynamically adapt to novel partners of different play styles and skill levels in the 2-player collaborative Overcooked game environment.
  arXiv  Detail & Related papers  (2023-05-26T07:53:12Z)
• PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination [52.991211077362586]
  We propose a policy ensemble method to increase the diversity of partners in the population. We then develop a context-aware method enabling the ego agent to analyze and identify the partner's potential policy primitives. In this way, the ego agent is able to learn more universal cooperative behaviors for collaborating with diverse partners.
  arXiv  Detail & Related papers  (2023-01-16T12:14:58Z)
• Conditional Imitation Learning for Multi-Agent Games [89.897635970366]
  We study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time. We propose a novel approach to address the difficulties of scalability and data scarcity. Our model learns a low-rank subspace over ego and partner agent strategies, then infers and adapts to a new partner strategy by interpolating in the subspace.
  arXiv  Detail & Related papers  (2022-01-05T04:40:13Z)
• On the Critical Role of Conventions in Adaptive Human-AI Collaboration [73.21967490610142]
  We propose a learning framework that teases apart rule-dependent representation from convention-dependent representation. We experimentally validate our approach on three collaborative tasks varying in complexity.
  arXiv  Detail & Related papers  (2021-04-07T02:46:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.