Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents

• URL: http://arxiv.org/abs/2510.25744v2
• Date: Thu, 30 Oct 2025 17:54:45 GMT
• Title: Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents
• Authors: Shannon Zejiang Shen, Valerie Chen, Ken Gu, Alexis Ross, Zixian Ma, Jillian Ross, Alex Gu, Chenglei Si, Wayne Chi, Andi Peng, Jocelyn J Shen, Ameet Talwalkar, Tongshuang Wu, David Sontag,
• Abstract summary: We argue for a shift from building and assessing task completion agents to developing collaborative agents.<n>We introduce collaborative effort scaling, a framework that captures how an agent's utility grows with increasing user involvement.
• Score: 48.95020665909723
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems, where human goals are often underspecified and evolve. We argue for a shift from building and assessing task completion agents to developing collaborative agents, assessed not only by the quality of their final outputs but by how well they engage with and enhance human effort throughout the problem-solving process. To support this shift, we introduce collaborative effort scaling, a framework that captures how an agent's utility grows with increasing user involvement. Through case studies and simulated evaluations, we show that state-of-the-art agents often underperform in multi-turn, real-world scenarios, revealing a missing ingredient in agent design: the ability to sustain engagement and scaffold user understanding. Collaborative effort scaling offers a lens for diagnosing agent behavior and guiding development toward more effective interactions.

Related papers

• Cross-Task Experiential Learning on LLM-based Multi-Agent Collaboration [63.90193684394165]
  We introduce multi-agent cross-task experiential learning (MAEL), a novel framework that endows LLM-driven agents with explicit cross-task learning and experience accumulation.<n>During the experiential learning phase, we quantify the quality for each step in the task-solving workflow and store the resulting rewards.<n>During inference, agents retrieve high-reward, task-relevant experiences as few-shot examples to enhance the effectiveness of each reasoning step.
  arXiv  Detail & Related papers  (2025-05-29T07:24:37Z)
• Who is Helping Whom? Analyzing Inter-dependencies to Evaluate Cooperation in Human-AI Teaming [13.263258837438045]
  We propose the concept of constructive interdependence as a key metric for evaluating cooperation in human-agent teams.<n>Our results demonstrate that although trained agents attain high task rewards, they fail to induce cooperative behavior.<n>Our analysis reveals that teaming performance is not necessarily correlated with task reward, highlighting that task reward alone cannot reliably measure cooperation.
  arXiv  Detail & Related papers  (2025-02-10T19:16:20Z)
• Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration [50.657070334404835]
  Collaborative Gym is a framework enabling asynchronous, tripartite interaction among agents, humans, and task environments.<n>We instantiate Co-Gym with three representative tasks in both simulated and real-world conditions.<n>Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance.
  arXiv  Detail & Related papers  (2024-12-20T09:21:15Z)
• A Survey on Complex Tasks for Goal-Directed Interactive Agents [60.53915548970061]
  This survey compiles relevant tasks and environments for evaluating goal-directed interactive agents. An up-to-date compilation of relevant resources can be found on our project website.
  arXiv  Detail & Related papers  (2024-09-27T08:17:53Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors [93.38830440346783]
  We propose a multi-agent framework framework that can collaboratively adjust its composition as a greater-than-the-sum-of-its-parts system. Our experiments demonstrate that framework framework can effectively deploy multi-agent groups that outperform a single agent. In view of these behaviors, we discuss some possible strategies to leverage positive ones and mitigate negative ones for improving the collaborative potential of multi-agent groups.
  arXiv  Detail & Related papers  (2023-08-21T16:47:11Z)
• Open-Ended Learning Leads to Generally Capable Agents [12.079718607356178]
  We define a universe of tasks within an environment domain and demonstrate the ability to train agents that are capable across this vast space and beyond. The resulting space is exceptionally diverse in terms of the challenges posed to agents, and as such, even measuring the learning progress of an agent is an open research problem. We show that through constructing an open-ended learning process, which dynamically changes the training task distributions and training objectives such that the agent never stops learning, we achieve consistent learning of new behaviours.
  arXiv  Detail & Related papers  (2021-07-27T13:30:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.