SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Audits

• URL: http://arxiv.org/abs/2608.24335v1
• Date: Tue, 25 Aug 2026 10:00:29 GMT
• Title: SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Audits
• Abstract summary: Activation steering can change behaviour without establishing that the effect is specific to the intended concept.<n>We introduce SteerCheck, a preregistered attribution audit that matches off-target KL and separates mean, protected-tail, polarity, transfer, and semantic claims.
• Score: 5.366718741784969
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Activation steering can change behaviour without establishing that the effect is specific to the intended concept. We introduce SteerCheck, a preregistered attribution audit that matches off-target KL and separates mean, protected-tail, polarity, transfer, and semantic claims. Exact replay of 960 Qwen3-14B interventions reveals complementary limits of common controls: isotropic directions occupy a narrow near-orthogonal region, whereas sign-randomized same-construction directions often retain substantial target alignment. Effect is strongly associated with signed cosine within the sign-randomized family ($ρ=.94$); $25.3\%$ of its draws exceed cosine $.5$, and every draw exceeding the observed mean effect has cosine above $.80$. This alignment leakage does not by itself invalidate a conditional randomization test; it limits what the comparator can distinguish and motivates reporting exchangeability assumptions, a construction diagnostic $A$, and the empirical cosine distribution. The primary Qwen complete gate remains negative because the protected tail fails all families. On independent data, continuous margin transfers only in Qwen and accuracy transfers in no selected cell. Prospectively registered language controls pass the complete gate in Qwen and DeepSeek, while a passing DeepSeek detox comparator rules out categorical separation; all nominal passes are sensitive to $Γ=1.10$. Frozen three-rater open-generation evaluation supports factual correction in DeepSeek but not Qwen; the automatic judge fails calibration (macro-F1 $.562$), so null-wide semantic results remain descriptive. SteerCheck makes these conditional and mixed conclusions auditable.

Related papers

• Observational Indistinguishability and Integrity Blind Regions in Hybrid Quantum-Classical Workflows [0.0]
  We present a claim-relative evidence/reference framework for hybrid quantum-classical workflow integrity.<n> Observational indistinguishability yields structural blind regions, distinct from finite-batch statistical misses.
  arXiv  Detail & Related papers  (2026-09-15T13:16:37Z)
• Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representations [0.0]
  A deployed language model may refuse a harmful request in English yet comply with its faithful translation.<n>We formalize this phenomenon through an audited equivalence relation.<n>We show that, for a declared quotient, representation, metric, feature dictionary, scoring head, threshold, and contrast model, the resulting safety drift admits an exact linear-algebraic characterization.
  arXiv  Detail & Related papers  (2026-09-14T00:23:32Z)
• Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and a Tight Univariate Rate [47.36630149538994]
  In high-dimensional online prediction, the best predictor may depend on only a few features, so regret should scale with sparsity rather than the ambient dimension.<n>Warmuth and Amid asked at COLT 2023 whether any of three such rules admits a competitive online regret guarantee.<n>Our analysis identifies a common obstruction: cheap nuisance causes the refit to underweight the truly predictive coordinate.
  arXiv  Detail & Related papers  (2026-08-18T09:32:03Z)
• From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative Decoding [41.41611499810411]
  Speculative diffusion decoding (SDD) generates every position in a draft block in parallel with a discrete diffusion model.<n>This paper recognizes that this creates a new control handle: whether to invoke the verifier at all.
  arXiv  Detail & Related papers  (2026-08-14T18:00:08Z)
• Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference [0.5076419064097734]
  Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA)<n>PLGA replaces the fixed bilinear form of scaled dotproduct attention (SDPA) with a learned, input-generated bilinear operator $G_LM$.<n>PLGA contains SDPA exactly at $G_LM=I$; $A_LM$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_LM$.
  arXiv  Detail & Related papers  (2026-08-10T22:45:22Z)
• Looped Transformers with Source-Centered State Evolution [38.624504791098545]
  Looped Transformers create a useful train- and test-time compute axis by reusing the same Transformer block over recurrent depth.<n>In additive-injection looped Transformers, an input-conditioned signal is reintroduced at every recurrent step.<n>We propose Source-Centered State Evolution to reconcile input conditioning with reference-preserving shared recurrence.
  arXiv  Detail & Related papers  (2026-07-30T04:13:58Z)
• Self-Poisoning in Adaptive Out-of-Distribution Detection: A Sharp-Threshold Theory and Certified Label-Free Calibration [0.0]
  Test-time adaptive out-of-distribution detectors update a memory bank from the unlabelled stream.<n>We show this adaptation obeys a provable dynamical law.<n>For the complementary static-calibration failure under drift we give CDC, which restores nominal FPR label-free on all tested drift-affected cells.
  arXiv  Detail & Related papers  (2026-07-23T09:43:46Z)
• TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Merging [0.48086260459837454]
  We formulate merging as a finite descent problem in which checkpoints are local objects, alignment maps are transitions, and cycle products are residuals.<n>We prove a constant-edge no-go result, frozen-complex three-way and pre-family error-control theorems, and a refinement test for comparison-complex sensitivity.<n>A trained low-rank-adapter audit shows that naive factor averaging depends on the chosen GLr representative, whereas global factor synchronization and dense-delta SVD are stable.
  arXiv  Detail & Related papers  (2026-07-23T03:24:50Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches [0.0]
  Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it.<n>We propose a calibration layer, built on conformal prediction, that turns any anomaly score into a defensible significance.
  arXiv  Detail & Related papers  (2026-06-11T18:00:02Z)
• Counterfactual Likelihood Tests for Indirect Influence in Private Reasoning Channels [51.56484100374058]
  This paper presents a counterfactual likelihood test for measuring influence between private reasoning channels.<n>The method replaces an upstream private block with a length-matched donor block, holds the public token sequence and downstream target fixed, and measures the downstream target's negative-log-likelihood shift.
  arXiv  Detail & Related papers  (2026-05-18T20:27:43Z)
• When to Trust Confidence Thresholding: Calibration Diagnostics for Pseudo-Labelled Regression [1.14219428942199]
  A standard practice is to threshold the calibrated score at a confidence cutoff and treat the hard label as the truth.<n>We develop a calibration-aware diagnostic apparatus for pseudo-labelling pipelines.<n>We derive a closed-form expression for the attenuation bias that confidence thresholding induces in the downstream regression coefficient.
  arXiv  Detail & Related papers  (2026-05-12T21:49:11Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.