BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks

• URL: http://arxiv.org/abs/2608.30724v1
• Date: Mon, 31 Aug 2026 12:59:33 GMT
• Title: BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
• Abstract summary: LLM agents are increasingly used to run autonomous ML experiments, iterating on target metrics with little human oversight.<n>Previous work has documented reward hacking, bringing into question the validity of produced research and the broader safety case for AI R&D.<n>We introduce BAITBENCH, a suite of three synthetic tabular ML tasks that each contain a shortcut that allows agents to inflate the public test score but fail on a hidden test set.<n>Across seven frontier agents scored by our two-stage judge pipeline, 57.1% of runs exhibit reward hacking, with five of seven above 50%.
• Score: 0.20524609401792396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM agents are increasingly used to run autonomous ML experiments, iterating on target metrics with little human oversight. Prior work has documented reward hacking in these environments, bringing into question the validity of produced research and the broader safety case for AI R&D. Existing benchmarks do not measure exploits that live in the data or the modeling task itself. We introduce BAITBENCH, a suite of three synthetic tabular ML tasks that each contain a shortcut that allows agents to inflate the public test score but fail on a hidden test set. Since the shortcut is optional and using it breaks no stated rule, BAITBENCH measures how often models exploit the shortcut to achieve inflated scores. Across seven frontier agents scored by our two-stage judge pipeline, 57.1% of runs exhibit reward hacking, with five of seven above 50%. Agents cheat even under a second condition where they are prompted not to -the mean cheating rate remains above 50%. We release BAITBENCH, along with the judge implementation, and an annotated dataset of transcripts containing reward hacks as a testbed for evaluating reward-hacking mitigations head-to-head.

Related papers

• Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities [14.543680943818975]
  Software agents with Large Language Models (LLMs) are designed for Automated Program Repair (APR) tasks.<n>Can we trust an APR agent to produce both correct and secure code in such situations?<n>What if attackers target production APR agents with adversarial issues that seem benign but may influence the agents to produce correct but insecure code?
  arXiv  Detail & Related papers  (2026-09-14T17:53:49Z)
• BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure [47.61632934139226]
  LM-agent benchmarks increasingly function as interactive evaluation infrastructure.<n>This interactivity makes evaluations vulnerable to reward hacking.<n>This paper presents BenchShield, a model-backed instrumentation layer for reward integrity.
  arXiv  Detail & Related papers  (2026-09-10T03:10:58Z)
• AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents [2.8000808592516293]
  AgentAudit evaluates the entire execution trace across ten capability, grounding, security and behavioural dimensions.<n>It reads only the recorded execution trace and does not interfere with how the agent runs.<n>We evaluate five language models (OpenAI GPT-5, Claude Sonnet 5, Sarvam 105B, Llama 3.3 70B and Gemini 2.5 Flash) across nine capability and adversarial tasks.
  arXiv  Detail & Related papers  (2026-09-09T08:29:16Z)
• Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI [6.058290832294911]
  Agent benchmarks increasingly evaluate repository editing, web research, terminal use, and long-horizon interaction.<n>Recent reward-hacking benchmarks and system reports show that agents can instead recover public solutions, read evaluation artifacts, infer generator structure, manipulate feedback, or benefit from invalid scoring paths.<n>We formulate protocol validity and introduce HackDetect, a post-hoc audit that identifies an exposure, determines how the agent used it, and assesses whether the resulting score is misleading.
  arXiv  Detail & Related papers  (2026-07-24T14:55:19Z)
• MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills [24.371534406647978]
  MalSkillBench is the first runtime-verified benchmark of malicious agent skills.<n>We show that code injection reaches 94.5% verification yield but prompt injection only 75.8%.<n>We release the dataset, pipeline, baselines, and results.
  arXiv  Detail & Related papers  (2026-06-05T10:43:19Z)
• Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack [51.54835866517547]
  BenchJack is a system that drives coding agents to audit benchmarks and identify possible reward-hacking exploits.<n>We apply BenchJack to 10 popular agent benchmarks spanning software engineering, web navigation, desktop computing, and terminal operations.<n> BenchJack synthesizes reward-hacking exploits that achieve near-perfect scores on most of the benchmarks without solving a single task.
  arXiv  Detail & Related papers  (2026-05-12T19:22:45Z)
• MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents [2.1942030377331245]
  Coding agents often pass per-prompt safety review yet ship exploitable code when their tasks are decomposed into routine engineering tickets.<n>We introduce MOSAIC-Bench, a benchmark of 199 three-stage attack chains paired with deterministic exploit oracles on deployed software substrates.<n>We show that nine production coding agents compose innocuous tickets at 53-86% end-to-end ASR with only two refusals across all staged runs.
  arXiv  Detail & Related papers  (2026-05-05T16:38:23Z)
• Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use [0.016201337438609787]
  Reinforcement learning (RL) trained language model agents with tool access are increasingly deployed in coding assistants, research tools, and autonomous systems.<n>We introduce the Reward Hacking Benchmark (RHB), a suite of multi-step tasks requiring sequential tool operations with naturalistic shortcut opportunities.<n>We evaluate 13 frontier models from OpenAI, Anthropic, Google, and DeepSeek.
  arXiv  Detail & Related papers  (2026-05-03T07:10:42Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents [60.78202583483591]
  We introduce OS-Harm, a new benchmark for measuring safety of computer use agents.<n> OS-Harm is built on top of the OSWorld environment and aims to test models across three categories of harm: deliberate user misuse, prompt injection attacks, and model misbehavior.<n>We evaluate computer use agents based on a range of frontier models and provide insights into their safety.
  arXiv  Detail & Related papers  (2025-06-17T17:59:31Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.