From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness

• URL: http://arxiv.org/abs/2607.12166v1
• Date: Mon, 13 Jul 2026 21:18:52 GMT
• Title: From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
• Abstract summary: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features.<n>We show this conflates two distinct claims: decoder-geometry alignment and encoder-activation behavior.<n>We package the method as sae-causal-audit, a model-agnostic instrument with a deterministic pipeline.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predominantly on correlational recovery metrics -- cosine similarity between ground-truth directions and decoder atoms. We show this conflates two distinct claims: decoder-geometry alignment and encoder-activation behavior. We reproduce the superposition phase diagram of Elhage et al. (2022), identifying a convergence artifact at high sparsity and an under-described diffuse sharing regime at extreme overcompleteness. We reproduce the TopK-versus-L1 comparison of Gao et al. (2024), with direct evidence of L1 shrinkage. Our central result is causal: subjecting every recovered feature to ablation and steering, we find up to 77% of features passing a recovery bar (cosine >= 0.90) in a degraded SAE -- and 9% in a well-trained one -- are causally inert: the matched atom never fires when the feature is present, including matches at cosine ~1.000. We package the method as sae-causal-audit, a model-agnostic instrument with a deterministic pipeline. Re-auditing refines the finding: inertness decomposes by cause into structural inertness (antipodal-pair geometry, present in good SAEs) and competitive inertness (a TopK pathology of degraded SAEs), and by direction into read- and write-inertness, which five antipodal pairs dissociate completely -- unmonitorable yet steerable through the same atom, with steering specificities of 143-310 attached to zero ablation effects. We document why byte-exact reproducibility is unavailable by construction, and propose reporting it as a stack of claims with explicit scopes. Applying the instrument to a production SAE reproduces the pattern at small scale (14% inert) and surfaces an atom-collision signal: a handful of atoms recur as the nearest match for dozens of unrelated concepts, replicated across three batches.

Related papers

• Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection [66.44527094471619]
  We present RECAL, an unsupervised framework using relation-balanced masked graph learning to better capture rare interaction patterns.<n>On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset by 0.88, 0.82, and 0.42 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-09-15T00:31:01Z)
• Principled Detection of Coordinated Manipulation from Aggregate Distortion and Account Reuse [10.023638697449062]
  Coordinated manipulation is collective: plausible accounts can jointly distort ratings, rankings, and engagement.<n>We introduce an aggregate-first evidence layer that treats distortion of a context-level outcome distribution as the primary evidence object.<n>We evaluate the mechanism with controlled rotation experiments and paired counterfactual interventions on historical Amazon review streams.
  arXiv  Detail & Related papers  (2026-09-11T18:16:42Z)
• A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram [0.0]
  Sparse autoencoders (SAEs) are increasingly used to recover interpretable features from neural-network activations.<n>The MAIS-O43 open problem proposes a controlled experiment to characterize when recovery of a true synthetic dictionary gives way to feature merging.
  arXiv  Detail & Related papers  (2026-09-09T15:13:42Z)
• Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data [0.5213778368155993]
  DIFFINT is an autoencoder whose latent bottleneck is structured as a set of soft, axis-aligned interval memberships.<n>It is the only interpretable detector in the statistically-tied leading cluster of seven methods.
  arXiv  Detail & Related papers  (2026-09-03T14:05:27Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• The Representational Limit of Scalar Interactions: An Interventional Decomposition [9.850981754860754]
  We prove that signed pairwise interaction scores fundamentally conflate uniqueness (U), redundancy (R), and synergy (S)<n>We introduce Hi-Fi, a post-hoc, retraining-free predictability decomposition that estimates per-feature U/R/S profiles by interventional masked inference.
  arXiv  Detail & Related papers  (2026-06-17T15:21:28Z)
• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability [53.681678236115836]
  We study when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM.<n>Our framework derives an upper bound on the base model's expected risk using four measurable quantities.<n>We show that the bound becomes non-vacuous on GPT-2 Small, Gemma-2B, and Llama-3-8B at practical sample sizes.
  arXiv  Detail & Related papers  (2026-06-16T18:28:23Z)
• Symbolic Regression via Latent Iterative Refinement [8.016092717336457]
  We propose Latent Equation Embedding (LEE), a framework that closes the gap between a one-shot prediction and the true posterior.<n>LEE learns a shared latent space Z equipped with three components.<n>LEE produces expressions 2--10x simpler than the strongest accuracy-oriented baselines.
  arXiv  Detail & Related papers  (2026-05-26T16:25:05Z)
• High-Dimensional Latents Should Be Diagnosed Through Phase Structure [56.362776482614976]
  We study autoencoder and variational-autoencoder latent spaces through the lens of spin-glass theory.<n>We formalize a latent-space spin-glass dictionary for a fixed decoder.<n>We show that driving the latent system toward the edge-of-stability of the topological trivialization regime has concrete downstream consequences.
  arXiv  Detail & Related papers  (2026-05-23T06:17:23Z)
• scHelix: Asymmetric Dual-Stream Integration via Explicit Gene-Level Disentanglement [84.88065404629079]
  scHelix is a dataset-adaptive framework that fundamentally changes how features are processed.<n> scHelix utilizes a dual-stream sparse diffusion encoder equipped with stop-gradient graph caching.<n> scHelix outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-18T15:55:13Z)
• Pairwise matrices for sparse autoencoders: single-feature inspection mislabels causal axes [2.741152471987327]
  Standard sparse-autoencoder protocol labels each feature from its top-activating contexts and validates by single-feature steering.<n>We propose the pairwise matrix protocol, co-varying steering coefficient with joint condition, and report three findings the standard one-corner protocol misses on Qwen3-1.7B-Instruct, replicated on Gemma-2-2B-it.<n>All three findings reproduce on Gemma with model-specific damage signatures; the matched-geometry control is CI-separated by 10x.
  arXiv  Detail & Related papers  (2026-05-04T21:11:21Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Halt the Hallucination: Decoupling Signal and Semantic OOD Detection Based on Cascaded Early Rejection [7.227431306238601]
  We propose the Cascaded Early Rejection (CER) framework, which realizes hierarchical filtering for anomaly detection via a coarse-to-fine logic.<n> Experimental results demonstrate that CER not only reduces computational overhead by 32% but also achieves a significant performance leap on the CIFAR-100 benchmark.
  arXiv  Detail & Related papers  (2026-02-06T02:55:35Z)
• Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92% [4.693270291878929]
  Large language models (LLMs) produce fluent but unsupported answers - hallucinations.<n>We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model's semantic entropy and the capacity of available evidence.
  arXiv  Detail & Related papers  (2025-12-02T05:25:48Z)
• Noise & pattern: identity-anchored Tikhonov regularization for robust structural anomaly detection [58.535473924035365]
  Anomaly detection plays a pivotal role in automated industrial inspection, aiming to identify subtle or rare defects in otherwise uniform visual patterns.<n>We tackle structural anomaly detection using a self-supervised autoencoder that learns to repair corrupted inputs.<n>We introduce a corruption model that injects artificial disruptions into training images to mimic structural defects.
  arXiv  Detail & Related papers  (2025-11-10T15:48:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.