Calibrating Interpretability Instruments Before Trusting Their Verdicts

• URL: http://arxiv.org/abs/2609.14754v1
• Date: Sun, 13 Sep 2026 19:37:02 GMT
• Title: Calibrating Interpretability Instruments Before Trusting Their Verdicts
• Abstract summary: Causal claims about large language model (LLM) internals rest on measurements.<n>These measurements fail in specific, diagnosable ways that return a plausible number instead of an error.<n>This note documents six such failures from a causal interpretability program on refusal and moral representation.
• Score: 1.0829694003408499
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal claims about large language model (LLM) internals rest on measurements. Those might include a projection, a cosine, an ablation delta, or an interchange patch among others. These measurements fail in specific, diagnosable ways that return a plausible number instead of an error, so a broken instrument can easily read as a finding. A covariance-matched null can saturate until every direction looks typical, a per-head attribution can overshoot the true residual write threefold on reordered-normalization architectures, an interchange patch can go sign-chaotic because its outcome is pinned at a ceiling, or a read-from verdict can be an artifact of measuring past the layer where the model already decided. This note documents six such failures from a causal interpretability program on refusal and moral representation, spanning several papers and a four-model open-weight panel; each mode is established on one or two of the four. For each we give the tell that catches it and a protocol keyed to a detectable trigger (reordered normalization, massive activations, a low-dimensional decision channel), so we and readers can check whether a given setup is exposed. The discipline reduces to four moves: calibrate against a positive-control ladder, certify with an orthogonal cell, compute power before spending compute, and state every read-from verdict at a depth referenced to the model's commitment. The evidence is four architectures across three families within a single program; external replication across programs is future work.

Related papers

• Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representations [0.0]
  A deployed language model may refuse a harmful request in English yet comply with its faithful translation.<n>We formalize this phenomenon through an audited equivalence relation.<n>We show that, for a declared quotient, representation, metric, feature dictionary, scoring head, threshold, and contrast model, the resulting safety drift admits an exact linear-algebraic characterization.
  arXiv  Detail & Related papers  (2026-09-14T00:23:32Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Split the Labor: Separating Evidence Interpretation from Decision Aggregation [0.0]
  Systems that ask a language model to reach a conclusion from many sources usually conflate them into one prompt.<n>We propose a four-field evidence (hypothesis, reliability bucket, rationale, provenance) and show that fixing it determines both halves.<n>The fix is arithmetic rather than architectural, and applies to a class of rules beyond language models.
  arXiv  Detail & Related papers  (2026-08-14T17:24:55Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering [0.8594140167290097]
  A model's agreement across perturbed inputs is used as a label-free reliability signal.<n>Rerendering beats resampling on both accuracy and reliability.<n>Fine-tuning on the model's own cross-render consensus inverts.
  arXiv  Detail & Related papers  (2026-08-06T07:08:05Z)
• Encoder-Decoder Manifold Alignment for Idempotent Generation [51.959208893243776]
  We argue that a key reason for this failure is a geometric mismatch between the manifold learned by the encoder and decoder.<n>We propose a new training framework that explicitly closes this gap by forcing the encoder and decoder to learn consistent representations of the same underlying data manifold.
  arXiv  Detail & Related papers  (2026-06-21T02:14:37Z)
• A Universal Cliff and a Design Fingerprint: Cross-Section Defect Detection Under LLM Orchestration [0.0]
  Production language-model systems answer a request byspanning it across an invisible orchestration of worker agents.<n>We ask what this does to a class of defect no single worker can see.<n>We vary only the model -- ten systems across five generations from one developer and five providers from distinct alignment paradigms.
  arXiv  Detail & Related papers  (2026-05-25T05:09:48Z)
• Silent Commitment Failure in Instruction-Tuned Language Models: Evidence of Governability Divergence Across Architectures [0.0]
  We introduce governability -- the degree to which a model's errors are detectable before output commitment and correctable once detected.<n>We show benchmark accuracy does not predict governability, correction capacity varies independently of detection, and identical governance scaffolds produce opposite effects across models.<n>We propose a Detection and Correction Matrix classifying model-task combinations into four regimes: Governable, Monitor Only, Steer Blind, and Ungovernable.
  arXiv  Detail & Related papers  (2026-03-22T21:50:28Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• GLENet: Boosting 3D Object Detectors with Generative Label Uncertainty Estimation [70.75100533512021]
  In this paper, we formulate the label uncertainty problem as the diversity of potentially plausible bounding boxes of objects. We propose GLENet, a generative framework adapted from conditional variational autoencoders, to model the one-to-many relationship between a typical 3D object and its potential ground-truth bounding boxes with latent variables. The label uncertainty generated by GLENet is a plug-and-play module and can be conveniently integrated into existing deep 3D detectors.
  arXiv  Detail & Related papers  (2022-07-06T06:26:17Z)
• Tractable Inference in Credal Sentential Decision Diagrams [116.6516175350871]
  Probabilistic sentential decision diagrams are logic circuits where the inputs of disjunctive gates are annotated by probability values. We develop the credal sentential decision diagrams, a generalisation of their probabilistic counterpart that allows for replacing the local probabilities with credal sets of mass functions. For a first empirical validation, we consider a simple application based on noisy seven-segment display images.
  arXiv  Detail & Related papers  (2020-08-19T16:04:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.