Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemma

• URL: http://arxiv.org/abs/2607.20522v1
• Date: Thu, 09 Jul 2026 05:04:25 GMT
• Title: Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemma
• Abstract summary: This paper tests whether holonomy concentrates on active sparse-autoencoder (SAE) feature planes in Gemma 2 2B.<n>The prediction was falsified in reverse: active-feature planes carried less holonomy than matched mixed-feature controls.<n>The cause remains open, with activation-strength geometry, degree of feature engagement, dictionary geometry, matched-center displacement, activation-manifold proximity, and transport shear as live alternatives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper tests whether holonomy concentrates on active sparse-autoencoder (SAE) feature planes in Gemma 2 2B, a concrete operationalization of the broader semantic-concentration prediction. Holonomy is measured at the final-token layer-12 to layer-13 residual-stream readout by carrying a local frame around small loops using the instrument's restricted-Jacobian transport rule, then normalizing the resulting rotation by enclosed area. The design, materiality threshold, analysis, and verdict rules were preregistered and frozen before the analysed measurements were inspected. The prediction was falsified in reverse: active-feature planes carried less holonomy than matched mixed-feature controls, with an adjusted log contrast of -0.29439 and 95% interval [-0.43989, -0.14889]. A magnitude-only explanation was not supported in this design, while the three-way ordering across random, mixed-feature, and active-feature planes was undefined at matched magnitude because common support failed. Post-freeze diagnostics at the same readout supported the area law on a small validation subset, bounded matched-center displacement under a simple paired regression, and identified transport distortion as a live mechanism or confound. The result is therefore a narrow, auditable operational reversal, not a causal claim that meaning suppresses holonomy. The cause remains open, with activation-strength geometry, degree of feature engagement, dictionary geometry, matched-center displacement, activation-manifold proximity, and transport shear as live alternatives.

Related papers

• Mechanistic Tomography: Designed Measurement for Control-Oriented Interpretability [0.0]
  Mechanistic interpretability seeks quantities that models do not expose directly.<n>We formulate their shared measurement structure as mechanistic tomography.<n>This language gives a practical procedure: start with the least costly measurements, test on held-out interventions at the intended scale, calibrate simple mismatch, and expand the measurement family.
  arXiv  Detail & Related papers  (2026-08-19T18:02:03Z)
• Excitation-Supervised Closed-Loop Self-Calibration and Target Seeking for an Unknown-Pose Range-Bearing Relay [0.0]
  This paper supplies the closed-loop layer that governs identifiability.<n>An excitation-supervised algorithm is used to supervise an excitation-reset controller.<n>A decay-rate sweep shows that supervision matters when a fixed schedule's decay outruns the unknown time-to-adequate-excitation.
  arXiv  Detail & Related papers  (2026-08-12T19:00:07Z)
• Orientation, not magnitude: the causal structure of task-vector interference in merged language models [0.0]
  We show that task arithmetic works until it doesn't, and the field diagnoses why with magnitudes.<n>An exact decomposition of the layerwise flux shows it is dominated by amplifying transport of the existing cross-term.<n>The striking +-15% "universality" of naive bfloat16 generation turns out to be quantization roughness.
  arXiv  Detail & Related papers  (2026-08-12T08:40:24Z)
• Local verification cannot detect non-transportability: a cohomological theory of context preservation in agentic reasoning [0.0]
  Agentic AI systems routinely transport conclusions across biological, clinical and financial contexts.<n>We show that the emerging safeguard of local verification is structurally incomplete.<n>We quantify the distortion under unequal precision and supply the precision-whitened form that restores exactness.
  arXiv  Detail & Related papers  (2026-08-04T21:09:50Z)
• Robust Activation Map Rectification for Weakly Supervised Volumetric Segmentation: Temporal Coherence as a Free Lunch [31.156115062163412]
  Weakly supervised segmentation relies heavily on class activation maps (CAMs) to initially localize target regions.<n>Existing remedies typically introduce additional training stages or prototype learning, increasing computational cost and reducing robustness.<n>We propose a training-free prototype-free framework that rectifies unreliable CAMs by exploiting temporal and structural coherence in volumetric data as a free lunch.
  arXiv  Detail & Related papers  (2026-07-22T08:07:38Z)
• Estimating Rare Events in Language Models with Proper Evaluation [16.1901795614153]
  We introduce Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS)<n>GA-AMLS adapts rare-event Monte Carlo methods to the continuous activation space of language models.<n>We also propose the Shifted-Power Bregman (SPB) Loss, a proper scoring rule that remains finite for zero-estimates.
  arXiv  Detail & Related papers  (2026-07-20T19:07:43Z)
• Geometric Collapse: When Vision Models Fail to Verify Physical Causality [52.43819179934414]
  We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues.<n>With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity.<n>Across CNN/ViT/SSL depth predictors, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise.
  arXiv  Detail & Related papers  (2026-07-08T00:04:02Z)
• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability [53.681678236115836]
  We study when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM.<n>Our framework derives an upper bound on the base model's expected risk using four measurable quantities.<n>We show that the bound becomes non-vacuous on GPT-2 Small, Gemma-2B, and Llama-3-8B at practical sample sizes.
  arXiv  Detail & Related papers  (2026-06-16T18:28:23Z)
• Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches [0.0]
  Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it.<n>We propose a calibration layer, built on conformal prediction, that turns any anomaly score into a defensible significance.
  arXiv  Detail & Related papers  (2026-06-11T18:00:02Z)
• FreeBridge: Variational Schrödinger Bridges for Cellular Transition Dynamics [48.14051378654045]
  FreeBridge is a Schrdinger Bridge for single-cell transition modeling under endpoint-only supervision.<n>FreeBridge maintains competitive or improved endpoint fidelity and mechanism mechanism retention under unified evaluation protocol.
  arXiv  Detail & Related papers  (2026-06-09T16:46:59Z)
• Measuring the Symmetry--Data Exchange Rate [0.2538209532048867]
  We show that a wrong-group control with identical orbit size and matched compute is worse than no constraint.<n>Relative exchange rate beta_diff = 1.28 is consistent in sign and order of magnitude with the theoretical bootstrap 1.0.<n>The methodological contributions -- the relative-rate estimator that cancels the shared-difficulty confound, the wrong-group control, and a pre-specified failure taxonomy -- transfer to any inductive bias whose strength can be parameterised.
  arXiv  Detail & Related papers  (2026-05-31T08:17:40Z)
• Unsupervised Domain Shift Detection with Interpretable Subspace Attribution [48.553902624178995]
  Domain shifts are subtle differences in the probability distributions of datasets.<n>We identify these shifts using an algorithm designed to detect localised density anomalies in high-dimensional feature spaces.<n>We trace the domain shift to a small set of features, making the shift interpretable.
  arXiv  Detail & Related papers  (2026-05-15T12:58:00Z)
• A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations [0.0]
  We develop a framework for superposition in large language models (LLMs) that replaces the single-global-dictionary premise with a sheaf-theoretic atlas of local semantic charts.<n> Contexts are clustered into a stratified context complex; each chart carries a local feature space and a local information-geometric metric.<n>This yields a Fisher-weighted interference energy and three measurable obstructions to global interpretability.
  arXiv  Detail & Related papers  (2026-02-28T22:05:14Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• Localization Uncertainty Estimation for Anchor-Free Object Detection [48.931731695431374]
  There are several limitations of the existing uncertainty estimation methods for anchor-based object detection. We propose a new localization uncertainty estimation method called UAD for anchor-free object detection. Our method captures the uncertainty in four directions of box offsets that are homogeneous, so that it can tell which direction is uncertain.
  arXiv  Detail & Related papers  (2020-06-28T13:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.