Decorrelation Is Not Complementarity: Skill, Not Lineage, Governs Trusted-Monitor Ensembles

• URL: http://arxiv.org/abs/2608.16190v1
• Date: Mon, 17 Aug 2026 07:10:56 GMT
• Title: Decorrelation Is Not Complementarity: Skill, Not Lineage, Governs Trusted-Monitor Ensembles
• Abstract summary: We study 24 open-weight monitors spanning nine pretraining lineages and a 29x range of detection skill.<n>At matched member capability, cross-lineage panels detect no better.<n>No correlation-weighted selection beats picking the single best monitor out of sample.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Trusted monitoring has a cheap, trusted model score a stronger untrusted model's actions, and a diverse ensemble of them beats a single stronger monitor at matched cost. They are built by minimising average pairwise correlation, and that paper's twelve monitors shared one base model, leaving open what supplies the diversity. We study 24 open-weight monitors spanning nine pretraining lineages and a 29x range of detection skill (pAUC at 10 percent FPR, 0.028 to 0.803) on backdoored code. The metric used to build panels does not predict what a panel is for, and we can say why. Agreement on attack items splits into a shared-detectability signal component and an idiosyncratic error component, which predict ensemble gain with opposite sign (Spearman -0.25 and +0.26), so their sum, the metric actually used, predicts it barely at all (+0.05); the cancellation holds in 7 of 8 evaluations. Skill acts on signal (+0.53) while error stays flat (-0.01), which is why a monitor's own skill predicts its agreement with the pool (Spearman 0.84, n = 24, permutation p below 0.0001). Pretraining lineage is the obvious way to buy decorrelation, and it does not pay. At matched member capability, cross-lineage panels detect no better (permutation p = 0.13), and lineage barely moves the metric either (+0.064, p = 0.18). We report that against ourselves: on our own 22-monitor pool the same test read +0.104 at p = 0.037 until two monitors were added. An earlier pool topping out at pAUC 0.23 had already invalidated another analysis. Such a quantity is a property of the pool assembled. Panel gain over the best member falls monotonically with panel skill (-0.66 at k = 2, -0.70 at k = 3), and no correlation-weighted selection beats picking the single best monitor out of sample. Across six attacker models the gain result holds in all six, the agreement and cancellation results in five of six.

Related papers

• Peer-Voted LLM-Agent Stress Tests Find Feed-Induced Lexical Convergence but No Reliable Matched-Exposure Advantage for Distributed Sources [0.013714053458441644]
  We introduce PV-SST, a peer-voted social-platform testbed.<n>Experiment comprises 448 trials and 112 complete model-by-topic-by-seed blocks.
  arXiv  Detail & Related papers  (2026-08-20T12:01:19Z)
• Whose Gold? Annotator-Pool Disagreement Is Large at the Item Level, and Hidden by Small Leaderboards [0.0]
  We show that preference benchmarks are built by hiring annotators, and the identity of those annotators is treated as an implementation detail.<n>On the 2,885 MultiPref items where both pools are internally unanimous, so no tie-breaking convention is consulted at all.<n>Yet on both corpora the resulting model leaderboards are bit-identical: Kendall tau = 1.00 with zero of six models displaced.
  arXiv  Detail & Related papers  (2026-08-17T00:19:52Z)
• Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection [0.0]
  Benchmark contamination is diagnosed with n-gram overlap, likelihood-based membership inference, or canary strings.<n>We show the natural way to do this does not work, specify one that survives measurement, then find that the correction making it work carries more variance than the null it is tested against.
  arXiv  Detail & Related papers  (2026-08-12T23:27:20Z)
• Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk [0.0]
  Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters.<n>A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale.<n>We build such a system end to end and test whether the second half of the promise holds.
  arXiv  Detail & Related papers  (2026-08-08T13:22:14Z)
• Is Personalized Modality Weighting Actually Personalized? A Controlled Audit of Per-User Weighting Claims in Multimodal Recommenders [29.643383372337308]
  We show that a single global weight delivers nearly all of the content gain across three independent short-video corpora.<n>We propose reporting real-GM alongside real-shuf as a minimum evidentiary standard for personalization claims.
  arXiv  Detail & Related papers  (2026-08-06T06:52:38Z)
• Detecting and Discriminating Operator Misspecification in Hybrid PDE-Parameter Learning: a Reference-Free Instrument, with Discrimination Bounded In Sample [0.0]
  We build an instrument that reads from a single fit and with no oracle.<n>We report where the instrument is blind, a pre-registered negative where a neural estimator loses to Tikhonov-regularized inversion at recovery.
  arXiv  Detail & Related papers  (2026-08-04T21:20:44Z)
• One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence [41.99844472131922]
  A single human must audit $N$ LLM agents under a budget of $B ll N$ audits per round.<n>We model this as budgeted noisy inspection over a two-level Gaussian copula and locate the miscalibration threshold $*$ past which confidence-ranked auditing is emphworse than random.
  arXiv  Detail & Related papers  (2026-07-30T14:52:36Z)
• Fixed-Protocol Amortized MPS Tomography with Conformalized Predictive Uncertainty [31.082191748525137]
  Quantum state tomography is sample-starved, and the states one prepares live on a narrow, learnable manifold.<n>A prior estimate is already near-optimal, so high fidelity at few measurements'' can be family memorization rather than tomography.<n>We propose a emph-protocol amortized MPS estimator trained once with a gauge-invariant fidelity loss.
  arXiv  Detail & Related papers  (2026-07-13T08:57:23Z)
• Scaffold splits hide structural-frontier failures in ADMET models [25.408600489964726]
  We introduce a label-free structural-frontier split that reserves the sparsest and most physicochemically remote scaffold groups.<n>We evaluate it on six public experimental or curated ADMET tasks.
  arXiv  Detail & Related papers  (2026-07-12T12:25:50Z)
• Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models [29.673427534201654]
  We introduce Facet-Probe, a five-facet audit of multimodal large language models.<n>We find that none of the 18 MLLMs we audit are order-invariant.<n>We propose cross-ordering flip rate as a standard reporting axis for MLLMs.
  arXiv  Detail & Related papers  (2026-06-24T17:53:26Z)
• Bernoulli CUSUM and Bayes-Optimal Detection Ceilings for Trust Fraud in Sparse Rating Networks [0.0]
  Sequential trust detection in rating networks relies on continuous observation models that fail on real data.<n>On Bitcoin-OTC, 56% of ratings take a single value under standard mapping, breaking the distributional assumptions that parametric detectors require.
  arXiv  Detail & Related papers  (2026-06-03T16:56:15Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference [28.863461307044435]
  We propose an attack that aligns shuffled activations to a common permutation and exploits them to extract model weights.<n> Experiments on Pythia-70m and GPT-2 demonstrate that the proposed attack can align shuffled activations with mean squared errors ranging from $10-9$ to $10-6$.<n>With a query cost of approximately $1, the adversary can recover model weights with L1-norm differences ranging from $10-4$ to $10-2$ compared to the oracle weights.
  arXiv  Detail & Related papers  (2026-05-06T13:31:15Z)
• Adaptive Confidence Intervals in Efron's Gaussian Two-Groups Model [28.636700996382558]
  Motivated by noise-oblivious nature of adversaries, we study confidence intervals for the null location parameter $$ in Efron's Gaussian two-groups fraction model.<n>We characterize the minimax-optimal length among confidence intervals with a prescribed coverage level uniformly over the unknown proportion contamination and all noise-oblivious adversaries.
  arXiv  Detail & Related papers  (2026-04-28T22:40:54Z)
• Causal Understanding by LLMs: The Role of Uncertainty [43.87879175532034]
  Recent papers show LLMs achieve near-random accuracy in causal relation classification.<n>We investigate whether pretraining exposure to causal examples improves causal understanding.
  arXiv  Detail & Related papers  (2025-09-24T13:06:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.