Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

• URL: http://arxiv.org/abs/2607.20911v1
• Date: Thu, 23 Jul 2026 04:34:06 GMT
• Title: Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction
• Abstract summary: Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
• Score: 57.138342889101345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Tencent WorkBuddy Bench, a multi-domain evaluation suite for coding agents; this report documents its construction methodology, scoring protocol, and a cross-model leaderboard. At its core is a unified evaluation framework for constructing and running distribution-informed coding-agent tasks across four work domains - Code, Web, Office, and Security. Rather than adapting public issue text, every task is reverse-engineered from a real commit, pull request, or business scenario and rewritten as a short, colloquial, role-played request, so that a task's prompt is not recoverable by web-searching the underlying issue, pull request, or commit thread. Because the dataset is released openly - task directories, environment images, evaluation harness, tests, and reference solutions - contamination resistance rests on this construction together with dataset versioning rather than on secrecy. The four subsets - repository-level engineering, front-end development, office and business workflows, and red-/blue-team security - probe complementary facets of real work, each with its own verification style. All are packaged in a uniform task-directory format and run, under a uniform and reproducible protocol, on two agent harnesses (CodeBuddy Code and Claude Code); the full open release makes the benchmark reproducible end to end and directly auditable, since any third party can re-run each task and inspect its content. Because each subset uses a different scoring instrument, scores are not comparable across subsets and the suite reports no suite-wide average. We report a cross-model leaderboard across several model families.

Related papers

• DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows [69.12339622053588]
  We introduce DuMateBench, a real-session benchmark reconstructed from anonymized and privacy-screened user sessions.<n>The resulting benchmark comprises 200 tasks spanning 8 broad scenarios and 17 fine-grained capability categories.<n>We execute these tasks in isolated Docker containers injected with three forms of real-world environmental complexity: Insufficient, Unstable, and Noisy.
  arXiv  Detail & Related papers  (2026-08-27T02:36:23Z)
• One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows [2.281697288703173]
  Thinkingbox is a sandbox for tool agent-user interaction.<n>It provides isolated MCP-compatible tool sessions, complete execution traces, and outcome evaluation over terminal backend.<n> Thinkingboxbench contains 507 benchmarks and state-conditioned benchmarks across numerous scenarios.
  arXiv  Detail & Related papers  (2026-08-20T07:37:57Z)
• HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following [0.8998688639218773]
  Handbook_md is a benchmark of 65 agentic tasks modeled on how employees follow company handbooks.<n>Tasks span five domains (finance, medical billing, insurance, logistics, and HR) and 10 fictional companies.<n> Grading is fully deterministic: each task carries a rubric of programmatic criteria that check both that required actions occurred and that prohibited actions did not.
  arXiv  Detail & Related papers  (2026-07-28T07:58:07Z)
• AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning [52.23991730630292]
  Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge.<n>We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer.<n>We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window.
  arXiv  Detail & Related papers  (2026-06-23T12:57:18Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• Theory Under Construction: Orchestrating Language Models for Research Software Where the Specification Evolves [1.0312968200748116]
  Large language models can generate substantial code and draft research text, but research-software projects require more than either artifact alone.<n>We identify two LM-specific failure modes: hallucination accumulation and desynchronization.<n>We propose Comet-H, an iterative prompt that orchestrates ideation, implementation, evaluation, grounding, and paper-writing as coordinates of a single workspace state.
  arXiv  Detail & Related papers  (2026-04-29T21:28:17Z)
• ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents [77.22389710754452]
  We introduce bench, a benchmark for coworker agents built around multi-turn multi-day tasks.<n>The current release contains 100 tasks across 13 professional scenarios, executed against five stateful sandboxed services.<n>The strongest model reaches 75.8 weighted score, but the best strict Task Success is only 20.0%, indicating that partial progress is common.
  arXiv  Detail & Related papers  (2026-04-26T16:05:02Z)
• Code Broker: A Multi-Agent System for Automated Code Quality Assessment [0.0]
  We present Code Broker, a multi agent system built with Google Agent Development Kit ADK.<n>It analyses Python code from files, local directories, or GitHub repositories and generates actionable quality assessment reports.
  arXiv  Detail & Related papers  (2026-04-25T00:53:59Z)
• DRBench: A Realistic Benchmark for Enterprise Deep Research [81.49694432639406]
  DRBench is a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings.<n>We release 15 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance.
  arXiv  Detail & Related papers  (2025-09-30T18:47:20Z)
• CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents [49.68117560675367]
  Crab is the first benchmark framework designed to support cross-environment tasks.<n>Our framework supports multiple devices and can be easily extended to any environment with a Python interface.<n>The experimental results demonstrate that the single agent with GPT-4o achieves the best completion ratio of 38.01%.
  arXiv  Detail & Related papers  (2024-07-01T17:55:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.