Evaluating Generative Models as Interactive Emergent Representations of Human-Like Collaborative Behavior

• URL: http://arxiv.org/abs/2605.03855v2
• Date: Wed, 06 May 2026 09:55:40 GMT
• Title: Evaluating Generative Models as Interactive Emergent Representations of Human-Like Collaborative Behavior
• Abstract summary: Human-AI collaboration requires AI agents to understand human behavior for effective coordination.<n>This work examines whether embodied foundation model agents exhibit emergent collaborative behaviors indicating underlying mental models of their collaborators.
• Score: 1.4016147623265656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human-AI collaboration requires AI agents to understand human behavior for effective coordination. While advances in foundation models show promising capabilities in understanding and showing human-like behavior, their application in embodied collaborative settings needs further investigation. This work examines whether embodied foundation model agents exhibit emergent collaborative behaviors indicating underlying mental models of their collaborators, which is an important aspect of effective coordination. This paper develops a 2D collaborative game environment where large language model agents and humans complete color-matching tasks requiring coordination. We define five collaborative behaviors as indicators of emergent mental model representation: perspective-taking, collaborator-aware planning, introspection, theory of mind, and clarification. An automated behavior detection system using LLM-based judges identifies these behaviors, achieving fair to substantial agreement with human annotations. Results from the automated behavior detection system show that foundation models consistently exhibit emergent collaborative behaviors without being explicitly trained to do so. These behaviors occur at varying frequencies during collaboration stages, with distinct patterns across different LLMs. A user study was also conducted to evaluate human satisfaction and perceived collaboration effectiveness, with the results indicating positive collaboration experiences. Participants appreciated the agents' task focus, plan verbalization, and initiative, while suggesting improvements in response times and human-like interactions. This work provides an experimental framework for human-AI collaboration, empirical evidence of collaborative behaviors in embodied LLM agents, a validated behavioral analysis methodology, and an assessment of collaboration effectiveness.

Related papers

• CoMind: Understanding Collaborative Human Activity from Multiple Minds and Views [82.15362247351763]
  We introduce a novel egocentric and exocentric video dataset capturing real-world collaboration in cooking scenarios.<n>The dataset integrates multi-perspective video, high-quality audio, gaze tracking, and 3D scene and object scans, with annotations for shared attention to objects.<n>We establish benchmarks for Joint Attention Estimation, Socially Conditioned Object Interaction Anticipation, and Collaborative Handover Prediction.
  arXiv  Detail & Related papers  (2026-07-07T18:08:45Z)
• HRIBench: Benchmarking Interaction-Centric Human-Robot Collaboration [62.19372189311434]
  We introduce HRIBench, a diagnostic benchmark for intent-aware human-robot collaboration based on executable scenarios.<n>HRIBench represents collaborative tasks as structured scenario scripts that explicitly model agent roles, temporal dependencies, coordination constraints, and human behavior distributions.<n>Results show that current foundation robot policies struggle substantially in collaborative settings despite strong manipulation ability.
  arXiv  Detail & Related papers  (2026-07-05T09:15:35Z)
• CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments [32.043112677406064]
  CollabSim is a simulation framework that combines a theory-grounded definition of collaborative capabilities, controlled manipulation of interaction conditions, and action-level probing of agents' internal states.<n> Experiments show that CollabSim can capture condition effects, separate model performance patterns, and reveal task-dependent effects of agent design.
  arXiv  Detail & Related papers  (2026-06-04T17:06:22Z)
• Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration [54.18136393879849]
  We present ALMANAC, a dataset of action-level mental model ANnotations for Agent Collaboration.<n>ALMANAC contains 2,987 collaboration actions, each paired with theory-informed mental model annotations.<n>Our results demonstrate ALMANAC's utility in evaluating models' ability to simulate human collaborative behaviors.
  arXiv  Detail & Related papers  (2026-06-04T16:56:12Z)
• Measuring Successful Cooperation in Human-AI Teamwork: Development and Validation of the Perceived Cooperativity and Teaming Perception Scales [3.8688888722378674]
  We introduce two theoretically grounded scales: the Perceived Cooperativity Scale (PCS) and the Teaming Perception Scale (TPS)<n>The PCS captures an agent's perceived cooperative capability and practice within a single interaction sequence; the TPS captures the emergent sense of teaming arising from mutual contribution and support.<n>Both scales successfully differentiated between cooperation partners of varying cooperative quality and showed construct validity in line with expectations.
  arXiv  Detail & Related papers  (2026-04-27T13:28:59Z)
• How large language models judge and influence human cooperation [82.07571393247476]
  We assess how state-of-the-art language models judge cooperative actions.<n>We observe a remarkable agreement in evaluating cooperation against good opponents.<n>We show that the differences revealed between models can significantly impact the prevalence of cooperation.
  arXiv  Detail & Related papers  (2025-06-30T09:14:42Z)
• When Trust Collides: Decoding Human-LLM Cooperation Dynamics through the Prisoner's Dilemma [10.143277649817096]
  This study investigates human cooperative attitudes and behaviors toward large language models (LLMs) agents.<n>Results revealed significant effects of declared agent identity on most cooperation-related behaviors.<n>These findings contribute to our understanding of human adaptation in competitive cooperation with autonomous agents.
  arXiv  Detail & Related papers  (2025-03-10T13:37:36Z)
• Who is Helping Whom? Analyzing Inter-dependencies to Evaluate Cooperation in Human-AI Teaming [13.263258837438045]
  We propose the concept of constructive interdependence as a key metric for evaluating cooperation in human-agent teams.<n>Our results demonstrate that although trained agents attain high task rewards, they fail to induce cooperative behavior.<n>Our analysis reveals that teaming performance is not necessarily correlated with task reward, highlighting that task reward alone cannot reliably measure cooperation.
  arXiv  Detail & Related papers  (2025-02-10T19:16:20Z)
• Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration [50.657070334404835]
  Collaborative Gym is a framework enabling asynchronous, tripartite interaction among agents, humans, and task environments.<n>We instantiate Co-Gym with three representative tasks in both simulated and real-world conditions.<n>Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance.
  arXiv  Detail & Related papers  (2024-12-20T09:21:15Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, a framework for better data construction and model tuning.<n>For insufficient data usage, we incorporate strategies such as Chain-of-Thought prompting and anti-induction.<n>For rigid behavior patterns, we design the tuning process and introduce automated DPO to enhance the specificity and dynamism of the models' personalities.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Large Language Model-based Human-Agent Collaboration for Complex Task Solving [94.3914058341565]
  We introduce the problem of Large Language Models (LLMs)-based human-agent collaboration for complex task-solving. We propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process.
  arXiv  Detail & Related papers  (2024-02-20T11:03:36Z)
• MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming [27.911816995891726]
  We introduce MetaAgents, a social simulation framework populated with Large Language Models (LLMs)<n>We construct a job fair environment as a case study to scrutinize the team assembly and skill-matching behaviors of LLM-based agents.<n>Our evaluation demonstrates that LLM-based agents perform competently in making rational decisions to develop efficient teams.
  arXiv  Detail & Related papers  (2023-10-10T10:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.