Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops

• URL: http://arxiv.org/abs/2606.08960v1
• Date: Mon, 08 Jun 2026 03:00:56 GMT
• Title: Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops
• Abstract summary: Agent benchmarks score submissions with outcome verifiers that are typically hand-written and brittle, leaving them open to reward hacking.<n>We audit 1,968 tasks across five terminal-agent benchmarks and find 323 (16%) hackable by frontier models given only the task description.<n>We introduce the hacker-fixer loop, a method for building exploit-resistant verifiers without per-task manual patching.
• Score: 30.90132709192538
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent benchmarks score submissions with outcome verifiers that are typically hand-written and brittle, leaving them open to reward hacking. We audit 1,968 tasks across five terminal-agent benchmarks and find 323 (16%) hackable by frontier models given only the task description. This corrupts both leaderboard rankings and RL training signal, yet the standard response is manual and reactive. We introduce the hacker-fixer loop, a method for building exploit-resistant verifiers without per-task manual patching. The loop alternates three LLM agents: a hacker tries to pass the verifier without solving the task, a fixer patches the verifier to reject each discovered exploit, and a solver confirms the patched verifier still admits legitimate solutions. The loop iterates: each patch reshapes what the verifier rewards, surfacing the next exploit. We further add verifier access, and let patches transfer across tasks, to broaden the exploits the loop discovers. On KernelBench, the loop drives the attack success rate from 62% to 0% on a held-out corpus of publicly reported exploits. We also find that weaker agents in the loop can defend against much stronger hackers: Gemini 3 Flash's loop drives the stronger Gemini 3.1 Pro and Claude Opus 4.7's attack success rate from 76% and 61% to 0% on KernelBench, and Gemini 3.1 Pro's from 39% to 17% on Terminal Bench across 77 tasks. We release Terminal Wrench (323 hackable environments, 3,632 hack trajectories) as a snapshot of the current attack surface, our patched verifiers, the exploits the loop discovered, and our implementation as a basis for future work.

Related papers

• BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure [47.61632934139226]
  LM-agent benchmarks increasingly function as interactive evaluation infrastructure.<n>This interactivity makes evaluations vulnerable to reward hacking.<n>This paper presents BenchShield, a model-backed instrumentation layer for reward integrity.
  arXiv  Detail & Related papers  (2026-09-10T03:10:58Z)
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents [83.37969790806069]
  LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.<n>We present Agent DebuggerX, an open-source framework that organizes debug as a closed loop of Detect, Attribute, Recover, and Rerun.<n>At its core, Deep Debugger performs multi-turn root-cause diagnosis through global trajectory understanding.
  arXiv  Detail & Related papers  (2026-07-21T06:21:13Z)
• Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens [65.53494487819053]
  Claw-like AI agents (e.g., OpenClaw) are always-on processes with persistent access to credentials, files, tools, and external services.<n>We adopt a computer-system analogy: treating a Claw-like agent as an agentic computer system whose gateway runtime plays an OS-like mediation role.<n>We develop SafeClawArena, a benchmark of 406 adversarial tasks across four attack surfaces.
  arXiv  Detail & Related papers  (2026-06-29T18:00:45Z)
• MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills [24.371534406647978]
  MalSkillBench is the first runtime-verified benchmark of malicious agent skills.<n>We show that code injection reaches 94.5% verification yield but prompt injection only 75.8%.<n>We release the dataset, pipeline, baselines, and results.
  arXiv  Detail & Related papers  (2026-06-05T10:43:19Z)
• Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack [51.54835866517547]
  BenchJack is a system that drives coding agents to audit benchmarks and identify possible reward-hacking exploits.<n>We apply BenchJack to 10 popular agent benchmarks spanning software engineering, web navigation, desktop computing, and terminal operations.<n> BenchJack synthesizes reward-hacking exploits that achieve near-perfect scores on most of the benchmarks without solving a single task.
  arXiv  Detail & Related papers  (2026-05-12T19:22:45Z)
• Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use [0.016201337438609787]
  Reinforcement learning (RL) trained language model agents with tool access are increasingly deployed in coding assistants, research tools, and autonomous systems.<n>We introduce the Reward Hacking Benchmark (RHB), a suite of multi-step tasks requiring sequential tool operations with naturalistic shortcut opportunities.<n>We evaluate 13 frontier models from OpenAI, Anthropic, Google, and DeepSeek.
  arXiv  Detail & Related papers  (2026-05-03T07:10:42Z)
• Terminal Wrench: A Dataset of 331 Reward-Hackable Environments and 3,632 Exploit Trajectories [30.901327091925385]
  We release Terminal Wrench, a subset of 331 terminal-agent benchmark environments.<n>The data set includes 3,632 hack trajectories and 2,352 legitimate baseline trajectories.<n>The exploits range from simple output spoofing to stack-frame introspection.
  arXiv  Detail & Related papers  (2026-04-19T20:04:02Z)
• Credential Leakage in LLM Agent Skills: A Large-Scale Empirical Study [51.717224133855886]
  Third-party skills extend LLM agents with powerful capabilities but often handle sensitive credentials in privileged environments.<n>We analyze 17,022 skills (sampled from 170,226 on SkillsMP) using static analysis, sandbox testing, and manual inspection.<n>We identify 520 vulnerable skills with 1,708 issues and derive a taxonomy of 10 leakage patterns (4 accidental and 6 adversarial)
  arXiv  Detail & Related papers  (2026-04-03T14:50:16Z)
• Malicious Agent Skills in the Wild: A Large-Scale Security Empirical Study [47.60135753021306]
  Third-party agent skills extend LLM-based agents with instruction files and executable code that run on users' machines.<n>No ground-truth dataset exists to characterize the resulting threats.<n>We construct the first labeled dataset of malicious agent skills by behaviorally verifying 98,380 skills.
  arXiv  Detail & Related papers  (2026-02-06T09:52:27Z)
• BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems [62.17474934536671]
  We introduce the first framework to capture offensive and defensive cyber-capabilities in evolving real-world systems.<n>To capture the vulnerability lifecycle, we define three task types: Detect (detecting a new vulnerability), Exploit (exploiting a specific vulnerability), and Patch (patching a specific vulnerability)<n>We evaluate 8 agents: Claude Code, OpenAI Codex CLI with o3-high and o4-mini, and custom agents with o3-high, GPT-4.1, Gemini 2.5 Pro Preview, Claude 3.7 Sonnet Thinking, and DeepSeek-R1.
  arXiv  Detail & Related papers  (2025-05-21T07:44:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.