WorkBench Revisited: Workplace Agents Two Years On

• URL: http://arxiv.org/abs/2606.13715v1
• Date: Wed, 10 Jun 2026 21:21:35 GMT
• Title: WorkBench Revisited: Workplace Agents Two Years On
• Abstract summary: The best agent on WorkBench in March 2024, GPT-4, completed 43% of tasks and took an unintended harmful action, such as emailing the wrong person, on 26% of them.<n>We re-visit the benchmark in June 2026 and find that the best agent to date, Claude Opus 4.8, completes 89% and takes an unintended harmful action on 2.5%.
• Score: 1.2691047660244335
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The best agent on WorkBench in March 2024, GPT-4, completed 43% of tasks and took an unintended harmful action, such as emailing the wrong person, on 26% of them. We re-visit the benchmark in June 2026 and find that the best agent to date, Claude Opus 4.8, completes 89% and takes an unintended harmful action on 2.5%. Aside from this considerable progress in frontier agent performance, three things stand out. First, capability and safety go together on WorkBench rather than trade off, so the models that finish the most tasks also do the least unintended damage. Second, while several classes of error have been totally eliminated, frontier models still make some basic mistakes that occasionally result in irreversible harm, such as sending an email to the wrong person. Third, the rise of open-weight models has drastically lowered costs for a performance level that was previously only accessible to proprietary models, while frontier costs have stayed relatively stable. We release an updated version of the benchmark with data and code quality improvements, new model scores, and analysis of agent progress on WorkBench since 2024.

Related papers

• EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction [14.50182233464472]
  We introduce early outcome prediction, a complementary axis of efficiency that instead cuts cost within each task.<n>EarlyEval can eliminate 13%-26% of agent steps and up to 44.1% input tokens and 29.4% output tokens at 89%-97% prediction accuracy.
  arXiv  Detail & Related papers  (2026-09-02T16:15:18Z)
• ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents [23.994328225666663]
  Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API.<n>We define behavioral safety and introduce ActBench, a benchmark that evaluates such behavior risk from execution trajectories rather than final responses.<n>ActBench contains 600 cases from 213 scenarios, spanning 15 risk behaviors, six execution spaces, and 48 web-service APIs.
  arXiv  Detail & Related papers  (2026-08-10T11:45:03Z)
• Reasoning effort, not tool access, buys first-try reliability in agentic code generation: an observational study [0.0]
  90 independent agent runs built the same application, a real time retrospective board, from one detailed specification.<n>The runs spanned several model generations, two agent harnesses, two reasoning effort levels, a testing tool, and two design oriented prompts.<n>The testing tool raised cost by 42 to 68 percent without improving functional score or reliability, even on interface visible criteria.
  arXiv  Detail & Related papers  (2026-07-02T17:08:21Z)
• Knowledge Index of Noah's Ark [63.143852586221534]
  We introduce KINA, an 899-item benchmark across 261 fine-grained disciplines.<n>We prove a bonus-on-bar tournament weakly FOSD-dominates flat payment in released-review quality.<n>The top model, Gemini-3.1-Pro-Preview, reaches 53.17%, followed by Claude-Opus-4.6 at 49.92% and GPT-5.4 at 48.55%.
  arXiv  Detail & Related papers  (2026-06-03T17:06:49Z)
• JobBench: Aligning Agent Work With Human Will [121.39878639038716]
  JobBench evaluates AI agents on the that experts identify as high-priority for delegation.<n>JobBench covers 130 agentic tasks across 35 occupations.
  arXiv  Detail & Related papers  (2026-05-25T21:07:02Z)
• ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents [59.626170560327274]
  We present textbfClawForge, a generator-backed benchmark framework for executable command-line categories under state conflict.<n>We instantiate this framework as the ClawForge-Bench (17 scenarios, 6 ability categories)
  arXiv  Detail & Related papers  (2026-05-13T21:34:08Z)
• Agentic Frameworks for Reasoning Tasks: An Empirical Study [2.210207223867301]
  We empirically evaluate 22 widely used agentic frameworks across three reasoning benchmarks: BBH, GSM8K, and ARC.<n>We evaluated them under a unified setting, measuring reasoning accuracy, execution time, computational cost, and cross-benchmark consistency.<n>Poorer results were mainly caused by orchestration problems rather than reasoning limits.
  arXiv  Detail & Related papers  (2026-04-17T19:02:54Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling [12.42698406379544]
  AgentHER recovers lost training signal by adapting the Hindsight Experience principle to natural-language agent trajectories.<n>It converts discarded failures into high-quality SFT, DPO, and ShareGPT training data.<n>On WebArena Replay (Zhou et al., 2024) and ToolBench (Qin et al., 2024), AgentHER improves over success-only SFT by +7.1-11.7 pp across four model families.
  arXiv  Detail & Related papers  (2026-03-22T18:36:58Z)
• BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills [59.003563837981886]
  High quality bugs are key to training the next generation of language model based software engineering (SWE) agents.<n>We introduce a novel method for synthetic generation of difficult and diverse bugs.
  arXiv  Detail & Related papers  (2025-10-22T17:58:56Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.