ObserverBench: Testing Mechanistic Estimates for Intervention and Control

• URL: http://arxiv.org/abs/2609.03026v1
• Date: Wed, 02 Sep 2026 18:01:17 GMT
• Title: ObserverBench: Testing Mechanistic Estimates for Intervention and Control
• Abstract summary: Mechanistic interpretability is increasingly used to guide interventions such as activation steering, circuit removal, and safety monitoring.<n>We present ObserverBench, a benchmark framework for testing whether an internal estimator is adequate for the intervention, control, or safety task it directs.<n>The benchmark reports estimation accuracy separately from the loss caused by the chosen action.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mechanistic interpretability is increasingly used to guide interventions such as activation steering, circuit removal, and safety monitoring. Yet an internal estimate that is accurate on average can still choose a poor action. We present ObserverBench, a benchmark framework for testing whether an internal estimator---an observer---is adequate for the intervention, control, or safety task it directs. Each task fixes the model, information boundary, allowed actions, decision rule, held-out cases, and loss. The benchmark reports estimation accuracy separately from the loss caused by the chosen action. Theory and experiments show why both are needed. In closed-loop control, observer errors matter at the starting point and along directions the allowed intervention can reach. On circuit-intervention tasks in GPT-2-small and Qwen2.5-7B, pairwise observers predict unseen effects more accurately without always choosing better actions; observers trained on action loss choose lower-loss actions. In safety triage, a score that perfectly separates violations can allocate a fixed intervention budget poorly when violations have different costs. Across Qwen2.5-7B, Gemma-2-9B-it, and prospectively frozen Qwen3.5-9B APPS tasks, AUROC can rank monitors differently from deployment loss, and the best information source changes across models. Sparse SAE readouts also trail their layer-matched dense controls on the reported Qwen panels, under disclosed activation-density or checkpoint mismatches. ObserverBench provides fixed task contracts, runnable baselines, and table-based submissions for evaluating interpretability methods through the actions they enable.

Related papers

• Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling [65.03665015319457]
  We propose an action-class-oriented diagnostic framework that decomposes multi-turn failures into two modes: action-class miscalibration and action-execution failure.<n>We validate it on a panel of tool-calling models across multiple multi-turn benchmarks.
  arXiv  Detail & Related papers  (2026-09-01T09:08:15Z)
• The Intervention Gap in Latent World Models [0.0]
  Planning-time intervention fidelity is a distinct, measurable property of a learned world model.<n>Across released TD-MPC2 checkpoint sizes, episode return falls as an operator-error diagnostic on task observables grows.<n>A self-supervised world model trained without task signal preserves the same operator substantially better than a task-anchored model on the shared task.
  arXiv  Detail & Related papers  (2026-08-30T19:47:51Z)
• Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings [2.065887219290771]
  Chain-of-thought (CoT) monitoring is increasingly treated as an important safety layer for frontier reasoning models.<n>A complementary axis for CoT-monitor evaluations is implicit-influence settings, where the prompt contains no instruction to hide.<n>We introduce the first benchmark that directly compares CoT monitorability under the two regimes.
  arXiv  Detail & Related papers  (2026-08-05T11:59:20Z)
• Calibration Is Not Control: Why LLM-Agent Oversight Needs Intervention [14.376426170653707]
  Two trajectory prefixes can have the same risk estimate while requiring different actions, because one remains recoverable and the other does not.<n>We formalize this mismatch as target error and identify intervention advantage, the expected utility gain from intervening rather than continuing.
  arXiv  Detail & Related papers  (2026-06-19T13:08:17Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Agentic Control in Variational Language Models [0.0]
  We study whether a variational language model can support a minimal and measurable form of agentic control grounded in its own internal evidence.<n>Our model combines local variational hidden computation (EVE), a homeostatic latent regulator, structurally aware checkpoint retention and a calibrated uncertainty-aware controller operating on top of the retained model.
  arXiv  Detail & Related papers  (2026-04-14T09:47:53Z)
• When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents [50.5814495434565]
  This work makes the first effort to define and study misaligned action detection in computer-use agents (CUAs)<n>We identify three common categories in real-world CUA deployment and construct MisActBench, a benchmark of realistic trajectories with human-annotated, action-level alignment labels.<n>We propose DeAction, a practical and universal guardrail that detects misaligned actions before execution and iteratively corrects them through structured feedback.
  arXiv  Detail & Related papers  (2026-02-09T18:41:15Z)
• A Granular Study of Safety Pretraining under Model Abliteration [64.24346997570275]
  We study model abliteration, a lightweight projection technique designed to remove refusal-sensitive directions.<n>We issue 100 prompts with balanced harmful and harmless cases, classify responses as **Refusal** or **Non-Refusal** using multiple judges, and validate judge fidelity.<n>Our study produces a checkpoint-level characterization of which data-centric safety components remain robust under abliteration.
  arXiv  Detail & Related papers  (2025-10-03T07:01:45Z)
• Preemptive Detection and Correction of Misaligned Actions in LLM Agents [58.39520480675366]
  InferAct is a novel approach to detect misaligned actions before execution.<n>It alerts users for timely correction, preventing adverse outcomes.<n>InferAct achieves up to 20% improvements on Marco-F1 against baselines in misaligned action detection.
  arXiv  Detail & Related papers  (2024-07-16T15:24:44Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.