Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

• URL: http://arxiv.org/abs/2505.20254v1
• Date: Mon, 26 May 2025 17:31:36 GMT
• Title: Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs
• Abstract summary: This paper argues that mechanistic interpretability should prioritize feature consistency in SAEs.<n>We propose using the Pairwise Dictionary Mean Correlation Coefficient as a practical metric to operationalize consistency.
• Score: 34.52554840674882
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse Autoencoders (SAEs) are a prominent tool in mechanistic interpretability (MI) for decomposing neural network activations into interpretable features. However, the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs, undermining the reliability and efficiency of MI research. This position paper argues that mechanistic interpretability should prioritize feature consistency in SAEs -- the reliable convergence to equivalent feature sets across independent runs. We propose using the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as a practical metric to operationalize consistency and demonstrate that high levels are achievable (0.80 for TopK SAEs on LLM activations) with appropriate architectural choices. Our contributions include detailing the benefits of prioritizing consistency; providing theoretical grounding and synthetic validation using a model organism, which verifies PW-MCC as a reliable proxy for ground-truth recovery; and extending these findings to real-world LLM data, where high feature consistency strongly correlates with the semantic similarity of learned feature explanations. We call for a community-wide shift towards systematically measuring feature consistency to foster robust cumulative progress in MI.

Related papers

• Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs [60.80435138270317]
  We apply reinforcement learning with metacognitive feedback to the problem of faithful calibration.<n>Experiments show RLMF achieves generalizable, state-of-the-art FC on diverse tasks.<n> RLMF surpasses standard RL by up to 63% while enhancing models' ability to assess and express their own capability limits.
  arXiv  Detail & Related papers  (2026-06-30T17:56:01Z)
• Latent Performance Profiling of Large Language Models [47.009623327601226]
  We introduce Latent Performance Profiling (LPP) -- a framework that derives task-agnostic diagnostics from hidden activations and output distributions.<n>Unlike static accuracy scores, LPP provides stable, architecture-sensitive signatures across models of similar size.<n>We show that models with similar benchmark scores can exhibit contrasting latent profiles, such as differences in entropy or adaptability.
  arXiv  Detail & Related papers  (2026-05-28T14:41:26Z)
• What Is Actually Being Annotated? Inter-Prompt Reliability as a Measurement Problem in LLM-Based Social Science Labeling [8.119041322912375]
  This paper introduces Inter-Prompt Reliability (IPR), a framework for evaluating the stability of large language models (LLMs) across linguistically varied prompts.<n>We evaluate this framework on two tasks with distinct properties: TREC (interpretative) and Politifact (knowledge-anchored)
  arXiv  Detail & Related papers  (2026-04-02T02:35:25Z)
• Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective [60.45433515408158]
  We show that long Chain-of-Thought (CoT) serves as a decisive decision-maker for the top option but fails to function as a granular distribution calibrator for ambiguous tasks.<n>We observe a distinct "decoupled mechanism": while CoT improves distributional alignment, final accuracy is dictated by CoT content.
  arXiv  Detail & Related papers  (2026-01-06T16:26:40Z)
• Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders [8.188989044347595]
  We propose a Sparse Autoencoder-based framework for retrieving and steering semantically interpretable internal features.<n>Using the Big Five personality traits as a case study, we demonstrate that our method enables precise, bidirectional steering of model behavior.
  arXiv  Detail & Related papers  (2026-01-06T12:40:37Z)
• Interpretive Efficiency: Information-Geometric Foundations of Data Usefulness [0.0]
  Interpretability is central to trustworthy machine learning, yet existing metrics rarely quantify how effectively data support an interpretive representation.<n>We propose Interpretive Efficiency, a normalized, task-aware functional that measures the fraction of task-relevant information transmitted through an interpretive channel.
  arXiv  Detail & Related papers  (2025-12-06T08:11:22Z)
• SHAP Distance: An Explainability-Aware Metric for Evaluating the Semantic Fidelity of Synthetic Tabular Data [7.227194225143588]
  We introduce the SHapley Additive exPlanations (SHAP) Distance, a novel explainability-aware metric that is defined as the cosine distance between the global SHAP attribution vectors.<n>We analyze datasets that span clinical health records with physiological features, enterprise invoice transactions with heterogeneous scales, and telecom churn logs with mixed categorical-numerical attributes.<n>Our results show that the SHAP Distance captures feature importance shifts and underrepresented tail effects that the Kullback-Leibler divergence and Train-on-Synthetic-Test-on-Real accuracy fail to detect.
  arXiv  Detail & Related papers  (2025-11-17T03:47:47Z)
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics [72.00014675808228]
  Instability in Large Language Models evaluation process obscures true learning dynamics.<n>We introduce textbfMaP, a framework that integrates underlineMerging underlineand the underlinePass@k metric.<n>Experiments show that MaP yields significantly smoother performance curves, reduces inter-run variance, and ensures more consistent rankings.
  arXiv  Detail & Related papers  (2025-10-10T11:40:27Z)
• Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders [50.52694757593443]
  Existing SAE training algorithms often lack rigorous mathematical guarantees and suffer from practical limitations.<n>We first propose a novel statistical framework for the feature recovery problem, which includes a new notion of feature identifiability.<n>We introduce a new SAE training algorithm based on bias adaptation'', a technique that adaptively adjusts neural network bias parameters to ensure appropriate activation sparsity.
  arXiv  Detail & Related papers  (2025-06-16T20:58:05Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervals [0.3495246564946556]
  We propose a novel, model-agnostic uncertainty attribution (UA) method grounded in conformal prediction (CP)<n>We define cooperative games where CP interval properties-such as width and bounds-serve as value functions, we attribute predictive uncertainty to input features.<n>Our experiments on synthetic benchmarks and real-world datasets demonstrate the practical utility and interpretative depth of our approach.
  arXiv  Detail & Related papers  (2025-05-19T13:49:05Z)
• SMARTe: Slot-based Method for Accountable Relational Triple extraction [1.2200609701777907]
  Triple Extraction (RTE) is a fundamental task in Natural Language Processing (NLP)<n>We propose SMARTe: a Slot-based Method for Accountable Triple extraction.<n>We demonstrate that adding interpretability does not compromise performance.
  arXiv  Detail & Related papers  (2025-04-17T10:21:15Z)
• Semi-supervised Semantic Segmentation with Multi-Constraint Consistency Learning [81.02648336552421]
  We propose a Multi-Constraint Consistency Learning approach to facilitate the staged enhancement of the encoder and decoder.<n>Self-adaptive feature masking and noise injection are designed in an instance-specific manner to perturb the features for robust learning of the decoder.<n> Experimental results on Pascal VOC2012 and Cityscapes datasets demonstrate that our proposed MCCL achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-23T03:21:33Z)
• Similarity-Distance-Magnitude Universal Verification [0.0]
  We build SDM networks with uncertainty-aware verification and interpretability-by-exemplar as intrinsic properties.<n>We provide open-source software implementing these results.
  arXiv  Detail & Related papers  (2025-02-27T15:05:00Z)
• FeaKM: Robust Collaborative Perception under Noisy Pose Conditions [1.9626657740463982]
  We introduce FeaKM, a novel method that employs Feature-level Keypoints Matching to correct pose discrepancies among collaborating agents.<n>Our experimental results demonstrate that FeaKM significantly outperforms existing methods on the DAIR-V2X dataset.
  arXiv  Detail & Related papers  (2025-02-16T06:03:33Z)
• Are Your LLMs Capable of Stable Reasoning? [38.03049704515947]
  We introduce G-Pass@$k$, a novel evaluation metric that continuously assesses model performance across multiple sampling attempts.<n>We employ G-Pass@$k$ in conjunction with state-of-the-art large language models to provide comprehensive insights into their potential capabilities and operational consistency.
  arXiv  Detail & Related papers  (2024-12-17T18:12:47Z)
• TRACE: TRansformer-based Attribution using Contrastive Embeddings in LLMs [50.259001311894295]
  We propose a novel TRansformer-based Attribution framework using Contrastive Embeddings called TRACE. We show that TRACE significantly improves the ability to attribute sources accurately, making it a valuable tool for enhancing the reliability and trustworthiness of large language models.
  arXiv  Detail & Related papers  (2024-07-06T07:19:30Z)
• Federated Contrastive Learning for Personalized Semantic Communication [55.46383524190467]
  We design a federated contrastive learning framework aimed at supporting personalized semantic communication. FedCL enables collaborative training of local semantic encoders across multiple clients and a global semantic decoder owned by the base station. To tackle the semantic imbalance issue arising from heterogeneous datasets across distributed clients, we employ contrastive learning to train a semantic centroid generator.
  arXiv  Detail & Related papers  (2024-06-13T14:45:35Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Robust Learning Through Cross-Task Consistency [92.42534246652062]
  We propose a broadly applicable and fully computational method for augmenting learning with Cross-Task Consistency. We observe that learning with cross-task consistency leads to more accurate predictions and better generalization to out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-06-07T09:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.