Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models

• URL: http://arxiv.org/abs/2606.24952v1
• Date: Tue, 23 Jun 2026 08:07:51 GMT
• Title: Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models
• Abstract summary: We test what is the angle between the direction that best detects a behavior and the one that best causes it?<n>If detection implies control the cosine is near 1; otherwise it quantifies a detection gap.<n>The cosine is a weight-computable signature of the dissociation between knowing and steering, not a predictor of it.
• Score: 1.9247157750972368
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A central aspiration of mechanistic interpretability is controllability: if we know where a behavior is represented in a model's activations, we should be able to modify it. This rests on a hidden premise -- that the direction which detects a behavior and the direction which controls it are the same, or close. We test this geometrically: what is the angle between the direction that best detects a behavior and the one that best causes it? If detection implies control the cosine is near 1; otherwise it quantifies a detection-intervention gap. On Gemma 2-2B-it, output format (clean JSON vs markdown fencing) collapses both roles onto one axis. Hallucination does not: the model detects fake entities with perfect linear separability (AUC = 1.000 from layer 5), yet that direction sits at cos = 0.12 (about 83 degrees) from the direction producing a refusal -- a small, reproducible alignment, far from the cos = 1 that "detection is control" would require. A detector built from activations, with no chosen tokens, likewise fails to align (cos = -0.06). The gap generalizes: across four models from three families and two scales (1B-9B), cos stays in [0.12, 0.20], identical before and after instruction tuning (0.1197 vs 0.1200), placing its origin in pretraining. A 15-degree rotation toward the refusal direction partially bridges it -- 73% and 60% refusal on two held-out fake-entity categories at 1.8% false positives. We then ask whether this cosine predicts steerability, and it does not: detection is a high-dimensional class, not a single direction, and what separates the steerable case is functional, not readable from a static angle. The cosine is a weight-computable signature of the dissociation between knowing and steering, not a predictor of it.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Orientation, not magnitude: the causal structure of task-vector interference in merged language models [0.0]
  We show that task arithmetic works until it doesn't, and the field diagnoses why with magnitudes.<n>An exact decomposition of the layerwise flux shows it is dominated by amplifying transport of the existing cross-term.<n>The striking +-15% "universality" of naive bfloat16 generation turns out to be quantization roughness.
  arXiv  Detail & Related papers  (2026-08-12T08:40:24Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• Measuring Alignment-Induced Activation Shifts Correctly: A Template-Controlled Difference-in-Differences Protocol [0.0]
  We show the obvious way to form this matrix is confounded.<n>The aligned model is evaluated under a chat template the base model never saw.<n>We distill it into measurement recommendations for activation-difference studies of alignment.
  arXiv  Detail & Related papers  (2026-05-23T13:47:17Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal [6.908637308550535]
  Chain-of-thought assumes that generated reasoning reflects a model's internal computation.<n>We show this assumption is wrong in a specific, measurable way.<n>Models internally detect their own reasoning errors but outwardly express confidence in them.
  arXiv  Detail & Related papers  (2026-05-10T12:26:46Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steering [70.99933391739154]
  We focus on two recurring failure modes where the agent repeatedly reasons over information it already has, and where it issues tool calls without integrating recent observations or acquiring new evidence.<n>In this paper, we introduce TACT (Think-Act via activation Steering), to detect and mitigate agent drift in the residual stream before it surfaces as a behavioral failure.<n>In specific, we label trajectory steps as overthinking, overacting, or calibrated, and find that their hidden states can separate linearly along two *drift axes*, pointing from calibrated behavior toward each failure mode.
  arXiv  Detail & Related papers  (2026-05-07T10:24:27Z)
• Reliable Control-Point Selection for Steering Reasoning in Large Language Models [28.288321095634128]
  Steering vectors offer a training-free mechanism for controlling reasoning behaviors in large language models.<n>However, constructing effective vectors requires identifying genuine behavioral signals in the model's hidden states.<n>We show that current methods detect these behaviors through keyword matching in chain-of-thought traces, implicitly assuming that every detected boundary encodes a genuine behavioral signal.<n>We develop a probabilistic model that formalizes intrinsic reasoning behaviors as events with context-dependent trigger probabilities, and show that unstable boundaries dilute the steering signal.
  arXiv  Detail & Related papers  (2026-04-02T14:48:56Z)
• The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models [1.45309944076563]
  We characterize correctness representations across 9 models from 5 architecture families.<n>We validate causally through activation steering.<n> correctness signal exists internally but is not expressed in outputs.
  arXiv  Detail & Related papers  (2026-02-08T23:27:10Z)
• Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation [64.92152574895111]
  We propose a simple Orthogonal Jacobian Regularization (OroJaR) to encourage deep generative model to learn disentangled representations. Our method is effective in disentangled and controllable image generation, and performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-17T15:01:46Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)
• On the Arbitrary-Oriented Object Detection: Classification based Approaches Revisited [94.5455251250471]
  We first show that the boundary problem suffered in existing dominant regression-based rotation detectors, is caused by angular periodicity or corner ordering. We transform the angular prediction task from a regression problem to a classification one. For the resulting circularly distributed angle classification problem, we first devise a Circular Smooth Label technique to handle the periodicity of angle and increase the error tolerance to adjacent angles.
  arXiv  Detail & Related papers  (2020-03-12T03:23:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.