Training Deliberative Monitors for Black-Box Scheming Detection

• URL: http://arxiv.org/abs/2605.29601v1
• Date: Thu, 28 May 2026 08:40:32 GMT
• Title: Training Deliberative Monitors for Black-Box Scheming Detection
• Abstract summary: Action-only deliberative monitors are trained to detect scheming from agentic trajectories without accessing the monitored agent's reasoning or model internals.<n>Our method, inspired by deliberative alignment, uses a scheming specification to elicit structured rationales from a frontier teacher.<n>We show that applying our method to Qwen3.5-27B yields higher performance than all low-cost frontier models.
• Score: 3.441374221174101
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As autonomous agents become more capable of performing real-world tasks, distinguishing scheming behavior from benign task pursuit may become a central AI control problem. Existing monitors often rely on chain-of-thought access or internal activations, or use prompted frontier models, all of which can be unavailable, unreliable or expensive in deployment. In this work, we study action-only deliberative monitors: smaller open-weight models trained to detect scheming and sabotage from agentic trajectories without accessing the monitored agent's reasoning or model internals. Our method, inspired by deliberative alignment, uses a scheming specification to elicit structured rationales from a frontier teacher, filters them with a separate judge, and distills the highest-quality rationales into open-weight monitors with supervised fine-tuning and reinforcement learning. We train on five datasets, and evaluate across six out-of-distribution agentic misalignment benchmarks. We show that applying our method to Qwen3.5-27B yields higher performance than all low-cost frontier models as prompted monitors (Gemini 3.1 Flash-Lite, GPT-5.4 Nano, and Claude Haiku 4.5) and than Gemini 2.5 Pro, while also achieving lower marginal inference cost (token-metered USD per 1,000 evaluations). Stronger prompted frontier monitors (Gemini 3.1 Pro, GPT-5.4, Claude Sonnet 4.6, and Claude Opus 4.6) achieve higher performance but at roughly $16$--$34\times$ higher marginal inference cost. Several of our trained monitors are positioned on the empirical cost--performance Pareto frontier among the monitors we evaluate, providing practical low-cost, low-FPR alternatives to prompted frontier models.

Related papers

• MOLE: Detecting Insider Threats in AI Agents [24.05505413434997]
  MOLE is an open benchmark of 150 AI-operated accounts sharing 9 stateful services over 30 workdays.<n>Of 39 agent models, 72% complete most assigned harmful objectives and agent refusal does not predict completion.<n>MOLE enables comparison of 40 monitors across corpus generators, observability levels, and threats.
  arXiv  Detail & Related papers  (2026-09-07T03:04:31Z)
• Bootstrapped Monitoring: Leveraging Transparent Reasoning to Oversee Stronger AI Agents [1.5722704995548957]
  As frontier models grow more capable, the capabilities gap between trusted and untrusted models may render trusted models unreliable monitors.<n>We introduce emphbootstrapped monitoring, a protocol that addresses this by inserting a stronger, intermediate untrusted model with transparent chain-of-thought reasoning into the oversight chain.<n>Our results suggest that bootstrapped monitoring can extend the useful lifetime of trusted models in control as AI capabilities advance.
  arXiv  Detail & Related papers  (2026-06-10T12:24:29Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• $D^2$-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation-Aware Routing [63.49501120848927]
  Safety monitoring for large language models (D-LLMs) remains largely unexplored.<n>We propose $D2$-Monitor, a bi-level safety monitor for D-LLMs.<n>$D2$-Monitor adopts a lightweight probe as an always-on monitor to jointly estimate hesitation and perform base classification.
  arXiv  Detail & Related papers  (2026-05-25T14:22:21Z)
• Training on Documents About Monitoring Leads to CoT Obfuscation [4.971619873738039]
  Chain-of-thought (CoT) monitoring is one of the most promising tools for detecting model misbehavior.<n>We study whether monitor-aware models are capable of obfuscating their reasoning to evade detection.
  arXiv  Detail & Related papers  (2026-05-14T17:59:01Z)
• CoT-Guard: Small Models for Strong Monitoring [13.584850312226918]
  Monitoring the chain-of-thought (CoT) of reasoning models is a promising approach for detecting covert misbehavior (i.e., hidden objectives) in code generation tasks.<n>Large models (GPT-5, Gemini-3-Flash) can serve as effective CoT monitors, but they are expensive to deploy due to the lengthy reasoning traces and high API cost.<n>We propose a post-training pipeline combining supervised fine-tuning (SFT) and reinforcement learning (RL), where SFT narrows the gap for in-domain tasks.
  arXiv  Detail & Related papers  (2026-05-12T20:49:30Z)
• MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring [0.0]
  We introduce a red-teaming methodology that exposes harder-to-catch attacks for monitoring agents.<n>We find that frontier monitors often detect suspicious actions but fall for persuasion or fail to calibrate suspiciousness scores appropriately.<n>MonitoringBench is a benchmark of 2,644 attack trajectories for evaluating monitor capabilities and failure modes.
  arXiv  Detail & Related papers  (2026-05-10T18:04:31Z)
• ExecTune: Effective Steering of Black-Box LLMs with Guide Models [81.45879384560016]
  We study a class of systems, termed Guide-Core Policies (GCoP), in which a guide model generates a structured strategy that is executed by a black-box core model.<n>We formalize GCoP under a cost-sensitive utility objective and show that end-to-end performance is governed by guide-averaged executability.<n>We propose ExecTune, a principled training recipe that combines teacher-guided acceptance sampling, supervised fine-tuning, and structure-aware reinforcement learning.
  arXiv  Detail & Related papers  (2026-04-09T23:27:46Z)
• DeceptGuard :A Constitutional Oversight Framework For Detecting Deception in LLM Agents [0.0]
  We introduce DECEPTGUARD, a unified framework that systematically compares three monitoring regimes.<n>We introduce DECEPTSYNTH, a scalable synthetic pipeline for generating deception-positive and deception-negative agent trajectories.<n>Our monitors are optimized on 4,800 synthetic trajectories and evaluated on 9,200 held-out samples from DeceptArena.
  arXiv  Detail & Related papers  (2026-03-14T06:45:43Z)
• How does information access affect LLM monitors' ability to detect sabotage? [5.941142438950269]
  We study how information access affects LLM monitor performance.<n>We show that contemporary systems often perform better with less information.<n>We find that agents unaware of being monitored can be caught much more easily.
  arXiv  Detail & Related papers  (2026-01-28T23:01:31Z)
• Beyond Linear Probes: Dynamic Safety Monitoring for Language Models [67.15793594651609]
  Traditional safety monitors require the same amount of compute for every query.<n>We introduce Truncated Polynomials (TPCs), a natural extension of linear probes for dynamic activation monitoring.<n>Our key insight is that TPCs can be trained and evaluated progressively, term-by-term.
  arXiv  Detail & Related papers  (2025-09-30T13:32:59Z)
• GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks [75.50160982584943]
  We introduce a Process Reward Model that provides dense, step-by-step feedback to guide agents.<n>Gui-Shepherd is trained on a diverse large-scale data set of $52$k interactions.<n>We are the first to conduct a systematic study of process supervision in GUI agents.
  arXiv  Detail & Related papers  (2025-09-28T08:35:16Z)
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation [56.102976602468615]
  We show that we can monitor a frontier reasoning model, such as OpenAI o3-mini, for reward hacking in agentic coding environments.<n>We find that with too much optimization, agents learn obfuscated reward hacking, hiding their intent within the chain-of-thought.
  arXiv  Detail & Related papers  (2025-03-14T23:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.