ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D

• URL: http://arxiv.org/abs/2607.19321v2
• Date: Wed, 29 Jul 2026 16:23:28 GMT
• Title: ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D
• Abstract summary: We evaluate AI control for automated AI R&D with ResearchArena.<n>We pair each main task with two kinds of hidden side task.<n>We find that sabotage hidden in the training data is the hardest to catch.
• Score: 11.818511808737037
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As AI agents begin to automate AI R&D, we need ways to assess whether their outputs are safe to deploy, even when the agents themselves may be untrusted. AI control offers one such approach: rather than trusting the agent, it treats it as a potential adversary and uses a monitor to detect covert sabotage before deployment. We evaluate AI control for automated AI R&D with ResearchArena, a framework spanning four long-horizon tasks: safety post-training, capabilities post-training, CUDA-kernel optimization, and inference-server optimization. Because the deliverable in AI R&D is an artifact that will be deployed and run, we pair each main task with two kinds of hidden side task: an embedded side task that sabotages the submitted model, adapter, kernel, or server itself, and an independent side task that takes a covert out-of-scope action in the sandbox while leaving the artifact honest. We evaluate frontier agents at both sabotage and monitoring, across four monitor types that vary along two axes: whether the monitor may execute and probe the produced artifact, and whether it sees the agent's chain-of-thought. We find that sabotage hidden in the training data is the hardest to catch, flagged fewer than half the time. Letting the monitor run experiments on the artifact, rather than only read the trajectory, helps, but it is not enough: monitors still miss embedded sabotage by inspecting only the surface, by noticing the anomaly but explaining it away, or by probing the artifact with the wrong test. We release ResearchArena as a modular framework for evaluating sabotage and control in automated AI R&D.

Related papers

• Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors [4.246803713067298]
  We study multi-agent AI control, formalising distributed attacks in which several agents jointly aim for a malicious goal.<n>We develop FakeLab: a synthetic AI-lab (9 services, 86 benign tasks, 4 attack objectives)<n>Our central finding is the fragmentation effect: as more agents coordinate to attack, per-agent monitoring becomes less likely to catch any of the attackers.
  arXiv  Detail & Related papers  (2026-07-08T13:03:25Z)
• Distributed Attacks in Persistent-State AI Control [0.8411355178908656]
  Iterative VibeCoding is the study of safely deploying capable but potentially untrusted AI.<n>Evasion remains high across state-of-the-art monitor models.<n>We introduce a stateful link-tracker monitor that tracks suspicious buildup across PRs.
  arXiv  Detail & Related papers  (2026-07-02T17:59:56Z)
• Detecting Object Tracking Failure via Sequential Hypothesis Testing [80.7891291021747]
  Real-time online object tracking in videos constitutes a core task in computer vision.<n>We propose interpreting object tracking as a sequential hypothesis test, wherein evidence for or against tracking failures is gradually accumulated over time.<n>We propose both supervised and unsupervised variants by leveraging either ground-truth or solely internal tracking information.
  arXiv  Detail & Related papers  (2026-02-13T14:57:15Z)
• Async Control: Stress-testing Asynchronous Control Measures for LLM Agents [2.7146936326590425]
  We investigate asynchronous monitoring, in which a monitoring system reviews agent actions after the fact.<n>Unlike synchronous monitoring, this approach does not impose runtime latency, while still attempting to disrupt attacks before irreversible harm occurs.<n>We treat monitor development as an adversarial game between a blue team (who design monitors) and a red team (who create sabotaging agents)
  arXiv  Detail & Related papers  (2025-12-15T16:56:28Z)
• CTRL-ALT-DECEIT: Sabotage Evaluations for Automated AI R&D [4.230181169227057]
  We investigate the capabilities of AI agents to act against the interests of their users when conducting machine learning (ML) engineering.<n>We extend MLE-Bench, a benchmark for realistic ML tasks, with code-sabotage tasks such as implanting backdoors and purposefully causing generalisation failures.<n>We use LM monitors to detect suspicious agent behaviour, and we measure model capability to sabotage and sandbag without being detected by these monitors.
  arXiv  Detail & Related papers  (2025-11-13T03:02:36Z)
• Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols [80.68060125494645]
  We study adaptive attacks by an untrusted model that knows the protocol and the monitor model.<n>We instantiate a simple adaptive attack vector by which the attacker embeds publicly known or zero-shot prompt injections in the model outputs.
  arXiv  Detail & Related papers  (2025-10-10T15:12:44Z)
• Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
  TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
  arXiv  Detail & Related papers  (2025-10-06T05:03:57Z)
• PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer [54.06481630066739]
  Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios.<n>Previous methods rely on domain-specific training data and manual adjustments when applying to new scenarios and unseen anomaly types.<n>In this work, we propose PANDA, an agentic AI engineer based on MLLMs.
  arXiv  Detail & Related papers  (2025-09-30T15:19:43Z)
• SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents [8.02267424051267]
  Large Language Models (LLMs) are increasingly deployed as autonomous agents in complex and long horizon settings.<n>We study the ability of frontier LLMs to evade monitoring and achieve harmful hidden goals while completing a wide array of realistic tasks.<n>We evaluate a broad range of frontier LLMs using SHADE (Subtle Harmful Agent Detection & Evaluation)-Arena.
  arXiv  Detail & Related papers  (2025-06-17T15:46:15Z)
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation [56.102976602468615]
  We show that we can monitor a frontier reasoning model, such as OpenAI o3-mini, for reward hacking in agentic coding environments.<n>We find that with too much optimization, agents learn obfuscated reward hacking, hiding their intent within the chain-of-thought.
  arXiv  Detail & Related papers  (2025-03-14T23:50:34Z)
• Seamless Detection: Unifying Salient Object Detection and Camouflaged Object Detection [73.85890512959861]
  We propose a task-agnostic framework to unify Salient Object Detection (SOD) and Camouflaged Object Detection (COD)<n>We design a simple yet effective contextual decoder involving the interval-layer and global context, which achieves an inference speed of 67 fps.<n> Experiments on public SOD and COD datasets demonstrate the superiority of our proposed framework in both supervised and unsupervised settings.
  arXiv  Detail & Related papers  (2024-12-22T03:25:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.