Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery

• URL: http://arxiv.org/abs/2609.12591v1
• Date: Fri, 11 Sep 2026 08:44:13 GMT
• Title: Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery
• Abstract summary: We study how state features and update features interact to produce downstream residual features.<n>Ultimately, our results suggest that feature flow atlases can serve as diagnostics of representation-update mechanisms.
• Score: 0.20415910628419062
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models are increasingly adapted through fine-tuning, model editing, and alignment procedures while retaining previously acquired capabilities. Understanding the internal computations that support these adaptations is therefore becoming increasingly important for continual model evolution. Sparse autoencoders (SAEs) provide interpretable feature dictionaries for residual-stream activations and sublayer outputs, but it remains unclear how state features and update features interact to produce downstream residual features. In this work, we focus on MLP updates as a first test case. We construct a transition atlas of triples $s_k + u_j \rightarrow t_\ell$, where a residual-state feature and an MLP-update feature jointly predict a target residual feature, and validate candidate triples by ablating the decoded update feature. In a 20M-token Pythia-160M $L_7 \rightarrow L_8$ run, we find 38,125 strong ablation-effect transitions, but 88.0% have both state-target and update-target decoder cosine similarity below 0.7. As a preliminary cross-model check, a run of 20M-token Gemma-3-4B $L_{21} \rightarrow L_{22}$ causally validates only the top 30,000 ranked candidate triples by ablating the decoded update feature, and 53.6% of strong-effect triples have both state-target and update-target decoder cosine similarity below 0.7. The Gemma result is directionally consistent with Pythia, but weaker, since update-target cosine recovers many of the strongest Gemma effects and the run is not a full-atlas causal validation. Ultimately, our results suggest that feature flow atlases can serve as diagnostics of representation-update mechanisms and thereby inform tools for steering model updates. Future work will validate more complex patterns across layers, models, and SAE families.

Related papers

• A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram [0.0]
  Sparse autoencoders (SAEs) are increasingly used to recover interpretable features from neural-network activations.<n>The MAIS-O43 open problem proposes a controlled experiment to characterize when recovery of a true synthetic dictionary gives way to feature merging.
  arXiv  Detail & Related papers  (2026-09-09T15:13:42Z)
• SANE: State Anomaly Neutralization for Stable Extreme-Context Delta-Rule Models [53.86918766240095]
  Delta-Rule recurrent models maintain a fixed-size state, enabling $O(1)$ inference memory but potentially becoming unstable under extreme-context extrapolation.<n>We propose textbfState Anomaly Neutralization (SANE), which applies adaptive $tanh$ compression at chunk boundaries while preserving the intra-chunk parallel structure.
  arXiv  Detail & Related papers  (2026-08-23T10:41:07Z)
• SpikeWorld: Fast-State Adaptation for Frozen Spiking World Models [0.0]
  We introduce SpikeWorld, a 1.45M sparse spiking model jointly trained for heterogeneous sensory prediction, semantics, image-text binding and action-conditioned dynamics.<n>Joint optimization improves action next-state MSE by 17.10% while also improving multimodal prediction, semantic accuracy and image-text retrieval.
  arXiv  Detail & Related papers  (2026-08-07T18:57:06Z)
• Leakage-Robust Evaluation and Data-Scale Sensitivity of Attention-Enhanced Multi-Task Learning for Joint Fault Diagnosis and Remaining Useful Life Estimation [0.0]
  Multi-task deep learning models can be affected by how sliding-window sequences are split into training and test sets.<n>We show that naive splitting can inflate classification accuracy from a genuine 20-60 percent to 99.9 percent.<n>We introduce a chunk-based, leakage-audited splitting protocol and evaluate all models using five seeds, one-way ANOVA, and Tukey HSD tests.
  arXiv  Detail & Related papers  (2026-07-17T20:33:36Z)
• Perplexity Can Miss SAE Feature Damage Under Quantization [0.16921396880325779]
  Quantized models are judged acceptable when perplexity or downstream accuracy remains close to the full-precision original.<n>We test this by using a frozen sparse-autoencoder (SAE) as a fixed measurement basis.
  arXiv  Detail & Related papers  (2026-06-02T01:17:05Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification [0.0]
  We report a small, reproducible audit of sparse-autoencoder (SAE) features of GPT-2 small fire differently on failed versus successful trials.<n>On 300 prompts, GPT-2 small reaches 79.7% accuracy; 146 of the 24,576 features in the layer-8 residual-stream SAE release of Bloom (2024) clear a Holm-corrected significance threshold.<n>The strongest single correlate of failure -- feature 17,491, d=+2.93, Neuronpedia label 'cryptographic keys' -- is essentially silent except when the prompt's transferred object is 'the keys'
  arXiv  Detail & Related papers  (2026-05-21T16:55:27Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses [57.20181537213498]
  Agentic Harness Engineering (AHE) is a closed loop that automates harness engineering.<n>AHE addresses challenges through three matched observability pillars.<n>Ten AHE lift pass@1 on Terminal-Bench 2 from 69.7% to 77.0%, surpassing the human-designed harness Codex-CLI.<n>On SWE-bench-verified it tops aggregate success at 12% fewer tokens than the seed, and on Terminal-Bench 2 it yields +5.1 to +10.1pp cross-family gains.
  arXiv  Detail & Related papers  (2026-04-28T16:55:02Z)
• AlphaApollo: Orchestrating Foundation Models and Professional Tools into a Self-Evolving System for Deep Agentic Reasoning [110.57865233597762]
  We present AlphaApollo, a self-evolving agentic reasoning system.<n>It aims to address two bottlenecks in foundation model (FM) reasoning-limited model-intrinsic capacity and unreliable test-time computation.<n>AlphaApollo orchestrates multiple models with professional tools to enable deliberate, verifiable reasoning.
  arXiv  Detail & Related papers  (2025-10-05T15:42:24Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.