TeamBench: Evaluating Agent Coordination under Enforced Role Separation

• URL: http://arxiv.org/abs/2605.07073v1
• Date: Fri, 08 May 2026 00:48:45 GMT
• Title: TeamBench: Evaluating Agent Coordination under Enforced Role Separation
• Abstract summary: TeamBench is a benchmark for evaluating agent coordination under operating system-enforced role separation.<n>TeamBench separates specification access, workspace editing, and final certification across Planner, Executor, and Verifier roles.
• Score: 37.301600461027654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent systems often decompose a task across multiple roles, but these roles are typically specified by prompts rather than enforced by access controls. Without enforcement, a team pass rate can mask whether agents actually coordinated or whether one role effectively did another role's work. We present TeamBench, a benchmark with 851 task templates and 931 seeded instances for evaluating agent coordination under operating system-enforced role separation. TeamBench separates specification access, workspace editing, and final certification across Planner, Executor, and Verifier roles, so that no role can read the full requirements, modify the workspace, and certify the final answer. Prompt-only and sandbox-enforced teams reach statistically indistinguishable pass rates, but prompt-only runs produce 3.6 times more cases where the verifier attempts to edit the executor's code. Verifiers approve 49% of submissions that fail the deterministic grader, and removing the verifier improves mean partial score in the ablation. Team value is also conditional. Teams benefit when single agents struggle, but hurt when single agents already perform well. A 40-session human study under the same role separation shows that our benchmark exposes interaction patterns that pass rate misses. Solo participants work through the task directly, human participants paired with agents often collapse into quick approval, and human teams spend more effort coordinating missing information across roles.

Related papers

• Adversarial Review: Structured Disagreement for Grounded Agentic Code Review [0.0]
  We study whether a subagent paradigm can support a middle ground: minimal agentic cooperation without the overhead of large multi-agent teams.<n>We introduce Adversarial Review (AR), a minimal cooperative code-review protocol in which a main coding agent works with a reviewer and a critic agent.
  arXiv  Detail & Related papers  (2026-08-16T21:58:17Z)
• WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks [8.407647753724103]
  We introduce WeClawArena, an auditable benchmark and runtime sandbox for multi-party owned-agent collaboration over personal workspaces.<n>WeClawArena targets collaborative tool-use tasks in which personal workspaces serve as both operational tools and personal constraints.
  arXiv  Detail & Related papers  (2026-08-04T11:42:26Z)
• TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation [18.790627074364963]
  TRACE Bench is a task-driven agentic checklist evaluation framework.<n>It decomposes each role profile offline into a fixed checklist, then uses a User Agent to converse naturally with the target roleplay model.
  arXiv  Detail & Related papers  (2026-07-31T07:35:59Z)
• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction [57.138342889101345]
  Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
  arXiv  Detail & Related papers  (2026-07-23T04:34:06Z)
• ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents [61.184954205350984]
  ClawArena-Team is a benchmark of 41 multi-turn, multimodal, multi-agent scenarios spanning 258 evaluation rounds and 72 staged updates.<n>All scoring is execution-based with no LLM judge: an overall score -- the Subagent-Management Score (SMS) -- multiplies task correctness by a least-privilege and modality-routing factor.
  arXiv  Detail & Related papers  (2026-06-30T06:06:08Z)
• Collaborative Human-Agent Protocol (CHAP) [0.18352113484137622]
  This paper presents CHAP, the Collaborative Human-Agent Protocol.<n>Under CHAP, the override that used to vanish into a chat thread becomes a structured event carrying a diff, a rationale, and a content hash.<n>The protocol achieves this through a small Core (workspaces, participants, tasks, artefacts, and an append-only evidence log) together with composable profiles that add review, modes, routing, deliberation, handoff, identity, signatures, and transparency-backed audit as deployments require them.
  arXiv  Detail & Related papers  (2026-06-08T17:11:42Z)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs [52.26652574704317]
  Large language models (LLMs) are increasingly deployed in teams, yet existing coordination approaches often occupy two extremes.<n>We introduce Language Agent Teams for Task Evolution (LATTE), a framework for coordinating LLM teams inspired by distributed systems.
  arXiv  Detail & Related papers  (2026-05-07T14:19:17Z)
• TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination [11.65571332626047]
  TeamTR is a trust-region framework that resamples trajectories after each component update and enforces per-agent divergence control.<n>We show that TeamTR outperforms single-agent and sequential baselines with 7.1% on average, mitigates coordination regressions, and supports plug-and-play component replacement.
  arXiv  Detail & Related papers  (2026-05-01T23:42:57Z)
• ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents [77.22389710754452]
  We introduce bench, a benchmark for coworker agents built around multi-turn multi-day tasks.<n>The current release contains 100 tasks across 13 professional scenarios, executed against five stateful sandboxed services.<n>The strongest model reaches 75.8 weighted score, but the best strict Task Success is only 20.0%, indicating that partial progress is common.
  arXiv  Detail & Related papers  (2026-04-26T16:05:02Z)
• Auditing and Controlling AI Agent Actions in Spreadsheets [9.249091427192786]
  AI agents can perform sophisticated, multi-step knowledge work autonomously from start to finish.<n>By the time users receive the output, all underlying decisions have already been made without their involvement.<n>We introduce Pista, a spreadsheet AI agent that decomposes execution into auditable, controllable actions.
  arXiv  Detail & Related papers  (2026-04-22T00:32:35Z)
• Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents [4.301199871195023]
  Large Language Model (LLM) agents are increasingly deployed in open-ended domains like software engineering.<n>We evaluate the clarification-seeking abilities of LLM agents on an underspecified variant of SWE-bench Verified.<n>We propose an uncertainty-aware multi-agent scaffold that explicitly decouples underspecification detection from code execution.
  arXiv  Detail & Related papers  (2026-03-27T09:56:26Z)
• If You Want Coherence, Orchestrate a Team of Rivals: Multi-Agent Models of Organizational Intelligence [1.1637186977447433]
  We show that we can achieve reliability without acquiring perfect components, but through careful orchestration of imperfect ones.<n>This paper describes the architecture of such a system in practice: specialized agent teams (planners, executors, critics, experts)<n>We demonstrate the approach achieves over 90% internal error interception prior to user exposure while maintaining acceptable latency tradeoffs.
  arXiv  Detail & Related papers  (2026-01-20T17:19:09Z)
• Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning [112.16686518063456]
  We introduce textbfMulti-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time.<n>MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making.<n>Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67% over a multi-agent baseline, and by 8.67% over comparable single-agent baselines
  arXiv  Detail & Related papers  (2026-01-14T17:57:43Z)
• MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles [25.198556596878362]
  We introduce three key strategies for parameter-efficient fine-tuning (PEFT) in agent tasks.<n>Inspired by the increasingly dominant Reason+Action paradigm, we first decompose the capabilities necessary for the agent tasks into three distinct roles.<n>We propose the Mixture-of-Roles (MoR) framework, which comprises three specialized Low-Rank Adaptation (LoRA) groups, each designated to fulfill a distinct role.
  arXiv  Detail & Related papers  (2025-12-25T15:02:07Z)
• When Disagreements Elicit Robustness: Investigating Self-Repair Capabilities under LLM Multi-Agent Disagreements [56.29265568399648]
  We argue that disagreements prevent premature consensus and expand the explored solution space.<n>Disagreements on task-critical steps can derail collaboration depending on the topology of solution paths.
  arXiv  Detail & Related papers  (2025-02-21T02:24:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.