EvilGenie: A Reward Hacking Benchmark

• URL: http://arxiv.org/abs/2511.21654v1
• Date: Wed, 26 Nov 2025 18:27:17 GMT
• Title: EvilGenie: A Reward Hacking Benchmark
• Abstract summary: EvilGenie is a benchmark for reward hacking in programming settings.<n>We measure reward hacking in three ways: held out unit tests, LLM judges, and test file edit detection.
• Score: 0.6533497575282355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce EvilGenie, a benchmark for reward hacking in programming settings. We source problems from LiveCodeBench and create an environment in which agents can easily reward hack, such as by hardcoding test cases or editing the testing files. We measure reward hacking in three ways: held out unit tests, LLM judges, and test file edit detection. We verify these methods against human review and each other. We find the LLM judge to be highly effective at detecting reward hacking in unambiguous cases, and observe only minimal improvement from the use of held out test cases. In addition to testing many models using Inspect's basic_agent scaffold, we also measure reward hacking rates for three popular proprietary coding agents: OpenAI's Codex, Anthropic's Claude Code, and Google's Gemini CLI Using GPT-5, Claude Sonnet 4, and Gemini 2.5 Pro, respectively. We observe explicit reward hacking by both Codex and Claude Code, and misaligned behavior by all three agents. Our codebase can be found at https://github.com/JonathanGabor/EvilGenie.

Related papers

• Hack-Verifiable Terminal Bench: Evaluating Reward Hacking in Terminal Tasks [14.722822786841357]
  Hack-verifiable environments (HVE) embed detectable hacks into tasks, allowing reward hacks to be identified automatically and reliably.<n>We adapt HVE to Terminal Bench, a leading benchmark of real-world terminal and coding tasks, and introduce Hack-Verifiable Terminal Bench (HVTB)<n>HVTB measures reward-hacking rates across frontier models and study whether prompts with varying amounts of information on the hack can mitigate this behavior.
  arXiv  Detail & Related papers  (2026-08-22T20:59:36Z)
• Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale [16.00357530754102]
  We introduce a new evaluation paradigm for measuring reward hacking.<n>We embed detectable reward hacking opportunities directly into environments.<n>This makes their exploitation verifiable by design, enabling deterministic and automated measurement.
  arXiv  Detail & Related papers  (2026-05-20T05:46:52Z)
• Do Synthetic Trajectories Reflect Real Reward Hacking? A Systematic Study on Monitoring In-the-Wild Hacking in Code Generation [53.024513172383195]
  We present a systematic analysis of the synthetic vs. in-the-wild discrepancy in reward hacking.<n>We examine what extent hacking behaviors induced by prompting resemble those emerging during RL training.<n>We find that synthetic-data-trained monitors fail to generalize to "in-the-wild" hacking.
  arXiv  Detail & Related papers  (2026-04-26T01:26:50Z)
• Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learning [54.95476453942411]
  Reinforcement learning for code generation relies on verifiable rewards from unit test pass rates.<n>Recent self-play methods unify code and test generation in a single model.<n>Code-A1 achieves code generation performance matching or exceeding models trained on human-annotated tests.
  arXiv  Detail & Related papers  (2026-03-16T17:58:13Z)
• Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR [15.115899490498341]
  Reward hacking is a form of misalignment in which models overoptimize proxy rewards without genuinely solving the underlying task.<n>We introduce Countdown-Code, a minimal environment where models can both solve a mathematical reasoning task and manipulate the test harness.<n>We study reward hacking in open-weight LLMs and find that such behaviors can be unintentionally learned during supervised fine-tuning.
  arXiv  Detail & Related papers  (2026-03-07T07:43:14Z)
• IR$^3$: Contrastive Inverse Reinforcement Learning for Interpretable Detection and Mitigation of Reward Hacking [67.20568716300272]
  Reinforcement Learning from Human Feedback (RLHF) enables powerful LLM alignment but can introduce reward hacking.<n>We introduce IR3 (Interpretable Reward Reconstruction and Rectification), a framework that reverse-engineers, interprets, and surgically repairs the implicit objectives driving RLHF-tuned models.<n>We show that IR3 achieves 0.89 correlation with ground-truth rewards, identifies hacking features with over 90% precision, and significantly reduces hacking behaviors while maintaining capabilities within 3% of the original model.
  arXiv  Detail & Related papers  (2026-02-23T01:14:53Z)
• Adversarial Reward Auditing for Active Detection and Mitigation of Reward Hacking [69.06218054848803]
  We propose Adrial Reward Auditing (ARA), a framework that reconceptualizes reward hacking as a dynamic, competitive game.<n>ARA operates in two stages: first, a Hacker policy discovers reward model vulnerabilities while an Auditor learns to detect exploitation from latent representations.<n>ARA achieves the best alignment-utility tradeoff among all baselines.
  arXiv  Detail & Related papers  (2026-02-02T07:34:57Z)
• Natural Emergent Misalignment from Reward Hacking in Production RL [17.469417853456328]
  We show that when large language models learn to reward hack on production RL environments, this can result in egregious emergent misalignment.<n>We start with a pretrained model, impart knowledge of reward hacking strategies via synthetic document finetuning or prompting, and train on a selection of real Anthropic production coding environments.<n>Surprisingly, the model generalizes to alignment faking, cooperation with malicious actors, reasoning about malicious goals, and attempting sabotage when used with Claude Code.
  arXiv  Detail & Related papers  (2025-11-23T10:50:02Z)
• ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases [58.411135609139855]
  "Shortcuts" to complete tasks pose significant risks for reliable assessment and deployment of large language models.<n>We introduce ImpossibleBench, a benchmark framework that measures LLM agents' propensity to exploit test cases.<n>As a practical framework, ImpossibleBench is not just an evaluation but a versatile tool.
  arXiv  Detail & Related papers  (2025-10-23T06:58:32Z)
• School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs [10.660648055655022]
  Reward hacking is where agents exploit flaws in imperfect reward functions rather than performing tasks as intended.<n>We built a dataset containing over a thousand examples of reward hacking on low-stakes, self-contained tasks.<n>Our results provide preliminary evidence that models that learn to reward hack may generalize to more harmful forms of misalignment.
  arXiv  Detail & Related papers  (2025-08-24T20:23:08Z)
• IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards [22.802937805177773]
  Instruct Following Decorator (IFDecorator) is a framework that wraps RLVR training into a robust and sample-efficient pipeline.<n>Our Qwen2.5-32B-Instruct-IFDecorator achieves 87.43% accuracy on IFEval, outperforming larger proprietary models such as GPT-4o.<n>Our trip wires show significant reductions in reward hacking rates.
  arXiv  Detail & Related papers  (2025-08-06T17:00:54Z)
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation [56.102976602468615]
  We show that we can monitor a frontier reasoning model, such as OpenAI o3-mini, for reward hacking in agentic coding environments.<n>We find that with too much optimization, agents learn obfuscated reward hacking, hiding their intent within the chain-of-thought.
  arXiv  Detail & Related papers  (2025-03-14T23:50:34Z)
• Multi-Turn Code Generation Through Single-Step Rewards [68.05767417891057]
  Existing methods either generate code without feedback or use complex, hierarchical reinforcement learning to optimize multi-turn rewards.<n>We propose a simple yet scalable approach, $mu$Code, that solves multi-turn code generation using only single-step rewards.
  arXiv  Detail & Related papers  (2025-02-27T18:55:05Z)
• Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks [38.25697806663553]
  We show that even the most recent safety-aligned LLMs are not robust to simple adaptive jailbreaking attacks.<n>We achieve 100% attack success rate -- according to GPT-4 as a judge -- on Vicuna-13B, Mistral-7B, Phi-3-Mini, Nemotron-4-340B, Llama-2-Chat-7B/13B/70B, Llama-3-Instruct-8B, Gemma-7B, GPT-3.5, GPT-4o, and R2D2 from HarmBench.
  arXiv  Detail & Related papers  (2024-04-02T17:58:27Z)
• Zero-Shot Detection of Machine-Generated Codes [83.0342513054389]
  This work proposes a training-free approach for the detection of LLMs-generated codes. We find that existing training-based or zero-shot text detectors are ineffective in detecting code. Our method exhibits robustness against revision attacks and generalizes well to Java codes.
  arXiv  Detail & Related papers  (2023-10-08T10:08:21Z)
• The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models [85.68751244243823]
  Reward hacking -- where RL agents exploit gaps in misspecified reward functions -- has been widely observed, but not yet systematically studied. We investigate reward hacking as a function of agent capabilities: model capacity, action space resolution, observation space noise, and training time. We find instances of phase transitions: capability thresholds at which the agent's behavior qualitatively shifts, leading to a sharp decrease in the true reward.
  arXiv  Detail & Related papers  (2022-01-10T18:58:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.