Which Sparse Autoencoder Features Are Real? Model-X Knockoffs for False Discovery Rate Control

• URL: http://arxiv.org/abs/2511.11711v1
• Date: Wed, 12 Nov 2025 17:12:45 GMT
• Title: Which Sparse Autoencoder Features Are Real? Model-X Knockoffs for False Discovery Rate Control
• Abstract summary: We introduce Model-X knockoffs to SAE feature selection, using knock-off+ to control the false discovery rate (FDR)<n>Our method offers a re-producible and principled framework for reliable feature discovery by combining SAEs with multiple-testing-aware inference.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although sparse autoencoders (SAEs) are crucial for identifying interpretable features in neural networks, it is still challenging to distinguish between real computational patterns and erroneous correlations. We introduce Model-X knockoffs to SAE feature selection, using knock-off+ to control the false discovery rate (FDR) with finite-sample guarantees under the standard Model-X assumptions (in our case, via a Gaussian surrogate for the latent distribution). We select 129 features at a target FDR q=0.1 after analyzing 512 high-activity SAE latents for sentiment classification using Pythia-70M. About 25% of the latents under examination carry task-relevant signal, whereas 75% do not, according to the chosen set, which displays a 5.40x separation in knockoff statistics compared to non-selected features. Our method offers a re-producible and principled framework for reliable feature discovery by combining SAEs with multiple-testing-aware inference, advancing the foundations of mechanistic interpretability.

Related papers

• Learning What to Fail On: Failure-Mode Contextual Bandits for Adversarial Data Curation [14.914697247710622]
  We introduce a failure-aware adversarial retrieval-augmented framework for improving robustness in natural language understanding.<n>Our approach improves RoBERTa-base accuracy from 88.48% to 92.60% on SNLI, from 75.04% to 80.95% on ANLI, and from 54.67% to 71.99% on MultiNLI.<n>We demonstrate transfer to FEVER fact verification, achieving up to 79.86% FEVER score and 82.45% accuracy with RoBERTa-large.
  arXiv  Detail & Related papers  (2026-08-19T08:31:15Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Interpretable Computer Vision for Defect Detection in X-ray Tomography of Aerospace SiC/SiC Composites [0.0]
  p-ResNet-50 is a convolutional framework extended with a prototype layer that couples high detection accuracy with case-based explanations.<n>The framework is validated on an XCT patch dataset of approximately 12,000 patches extracted from four defect-rich SiC/SiC laboratory specimens.
  arXiv  Detail & Related papers  (2026-05-19T17:46:49Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Multi-Axis Trust Modeling for Interpretable Account Hijacking Detection [1.0152838128195467]
  This paper proposes a Hadith-inspired multi-axis trust modeling framework.<n>We translate five trust axes - long-term integrity (adalah), behavioral precision (dabt), contextual continuity (isnad), cumulative reputation, and anomaly evidence - into a compact set of 26 semantically meaningful behavioral features for user accounts.<n>We evaluate the framework on the CLUE-LDS cloud activity dataset with injected account hijacking scenarios.
  arXiv  Detail & Related papers  (2026-02-20T19:36:30Z)
• The Hypocrisy Gap: Quantifying Divergence Between Internal Belief and Chain-of-Thought Explanation via Sparse Autoencoders [0.0]
  We introduce the Hypocrisy Gap to quantify the divergence between a model's internal reasoning and its final generation.<n>By mathematically comparing an internal truth belief, derived via sparse linear probes, to the final generated trajectory in latent space, we quantify and detect a model's tendency to engage in unfaithful behavior.
  arXiv  Detail & Related papers  (2026-01-14T00:40:40Z)
• Deep Reinforcement Learning for Phishing Detection with Transformer-Based Semantic Features [0.0]
  Phishing is a cybercrime in which individuals are deceived into revealing personal information.<n>This study proposes a Quantile Deep Q-Network (QR-DQN) approach that integrates RoBERTa semantic embeddings with handcrafted lexical features.<n>A diverse dataset of 105,000 URLs was curated from PhishTank, OpenPhish, Cloudflare, and other sources.
  arXiv  Detail & Related papers  (2025-12-07T17:08:12Z)
• Distance Is All You Need: Radial Dispersion for Uncertainty Estimation in Large Language Models [13.41454380481593]
  We introduce bfRadial Dispersion Score (RDS), a simple, parameter-free, fully model-agnostic uncertainty metric.<n>RDS naturally extends to per-sample scoring, enabling applications such as best-of-$N$ selection and confidence-based filtering.
  arXiv  Detail & Related papers  (2025-12-04T00:53:49Z)
• Model Correlation Detection via Random Selection Probing [62.093777777813756]
  Existing similarity-based methods require access to model parameters or produce scores without thresholds.<n>We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test.<n>RSP produces rigorous p-values that quantify evidence of correlation.
  arXiv  Detail & Related papers  (2025-09-29T01:40:26Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Accelerated Test-Time Scaling with Model-Free Speculative Sampling [58.69141724095398]
  We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach.<n>We show that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding.<n>As a model-free approach, STAND can be applied to any existing language model without additional training.
  arXiv  Detail & Related papers  (2025-06-05T07:31:18Z)
• Human-in-the-Loop Feature Selection Using Interpretable Kolmogorov-Arnold Network-based Double Deep Q-Network [2.259291861960906]
  This study proposes a human-in-the-loop (HITL) feature selection framework integrated into a Double Deep Q-Network (DDQN) Our novel approach leverages simulated human feedback and distribution-based sampling, specifically Beta, to iteratively refine feature subsets per data instance. The KAN-DDQN achieved notable test accuracies of 93% on MNIST and 83% on FashionMNIST, outperforming conventional notable-DDQN models by up to 9%.
  arXiv  Detail & Related papers  (2024-11-06T08:13:09Z)
• FDINet: Protecting against DNN Model Extraction via Feature Distortion Index [25.69643512837956]
  FDINET is a novel defense mechanism that leverages the feature distribution of deep neural network (DNN) models. It exploits FDI similarity to identify colluding adversaries from distributed extraction attacks. FDINET exhibits the capability to identify colluding adversaries with an accuracy exceeding 91%.
  arXiv  Detail & Related papers  (2023-06-20T07:14:37Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)
• Feature Quantization Improves GAN Training [126.02828112121874]
  Feature Quantization (FQ) for the discriminator embeds both true and fake data samples into a shared discrete space. Our method can be easily plugged into existing GAN models, with little computational overhead in training.
  arXiv  Detail & Related papers  (2020-04-05T04:06:50Z)
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
  We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
  arXiv  Detail & Related papers  (2020-03-04T12:27:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.