Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

• URL: http://arxiv.org/abs/2606.24716v1
• Date: Tue, 23 Jun 2026 15:39:29 GMT
• Title: Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
• Abstract summary: We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts.<n>We validate this matching through targeted attribute perturbations.<n>Our framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs.
• Score: 12.535445487099393
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention-style evaluation in vision, we construct synCUB and synCOCO, synthetic benchmarks of paired images that differ in exactly one attribute. We introduce Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and consistently outperforms one-to-one baselines. For functional validation, we propose a Targeted Attribute Perturbation Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image-level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2 embeddings, we find that increased overcompleteness can reduce perturbation alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs. Code and datasets are available at https://github.com/JonasKlotz/sae-concept-eval.

Related papers

• SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models [63.28435103335999]
  We introduce SOCO, a new benchmark for Semantic Object Correspondence.<n>We show that vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories.<n>We also show that LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching.
  arXiv  Detail & Related papers  (2026-05-29T17:58:48Z)
• Interpretable Coreference Resolution Evaluation Using Explicit Semantics [35.71215348826291]
  Coreference resolution is evaluated using aggregate statistical metrics such as CoNLL-F1.<n>We introduce a semantically-enhanced evaluation framework for coreference resolution.<n>We show that our framework uncovers systematic weaknesses that remain obscured by aggregate metrics.
  arXiv  Detail & Related papers  (2026-05-11T14:20:16Z)
• SCALE: Semantic- and Confidence-Aware Conditional Variational Autoencoder for Zero-shot Skeleton-based Action Recognition [4.853241666510524]
  Zero-shot skeleton-based action recognition (ZSAR) aims to recognize action classes without any training skeletons from those classes.<n>We propose a lightweight and deterministic Semantic- and Confidence-Aware Listwise Energy-based framework that formulates ZSAR as class-conditional energy ranking.
  arXiv  Detail & Related papers  (2026-04-02T16:12:42Z)
• LUCID-SAE: Learning Unified Vision-Language Sparse Codes for Interpretable Concept Discovery [14.222802170483739]
  LUCID is a vision-language sparse autoencoder that learns a shared latent dictionary for image patch and text token representations.<n> LUCID yields interpretable shared features that support patch-level grounding, establish cross-modal neuron correspondence, and enhance robustness against the concept clustering problem.<n>Our analysis reveals that LUCID's shared features capture diverse semantic categories beyond objects, including actions, attributes, and abstract concepts.
  arXiv  Detail & Related papers  (2026-02-07T02:01:25Z)
• AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering [97.52852990265136]
  We introduce AQAScore, a backbone-agnostic evaluation framework that leverages the reasoning capabilities of audio-aware large language models.<n>We evaluate AQAScore across multiple benchmarks, including human-rated relevance, pairwise comparison, and compositional reasoning tasks.
  arXiv  Detail & Related papers  (2026-01-21T07:35:36Z)
• ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts [54.60525564599342]
  ConceptScope is a scalable and automated framework for analyzing visual datasets.<n>It categorizes concepts into target, context, and bias types based on their semantic relevance and statistical correlation to class labels.<n>It reliably detects known biases and uncovers previously unannotated ones.
  arXiv  Detail & Related papers  (2025-10-30T06:46:17Z)
• Evaluating SAE interpretability without explanations [0.7234862895932991]
  We adapt existing methods to assess the interpretability of sparse coders, with the advantage that they do not require generating natural language explanations as an intermediate step.<n>We compare the scores produced by our interpretability metrics with human evaluations across similar tasks and varying setups, offering suggestions for the community on improving the evaluation of these techniques.
  arXiv  Detail & Related papers  (2025-07-11T10:31:53Z)
• Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval [15.665567982431924]
  We propose a novel interpretability framework for Dense Passage Retrieval (DPR) models.<n>We generate natural language descriptions for each latent concept, enabling human interpretations of both the dense embeddings and the query-document similarity scores of DPR models.<n>We show that Concept-Level Sparse Retrieval (CL-SR) achieves high index-space and computational efficiency while maintaining robust performance across vocabulary and semantic mismatches.
  arXiv  Detail & Related papers  (2025-05-28T02:50:17Z)
• Post-hoc Probabilistic Vision-Language Models [54.05237186168399]
  Vision-language models (VLMs) have found remarkable success in classification, retrieval, and generative tasks.<n>We propose post-hoc uncertainty estimation in VLMs that does not require additional training.<n>Our results show promise for safety-critical applications of large-scale models.
  arXiv  Detail & Related papers  (2024-12-08T18:16:13Z)
• Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
  We propose a new evaluation method, SQC-Score. Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score. Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
  arXiv  Detail & Related papers  (2024-04-04T15:36:53Z)
• Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing [97.70862116338554]
  We investigate the problem of measuring interpretability of self-supervised representations. We formulate the latter as estimating the mutual information between the representation and a space of manually labelled concepts. We use our method to evaluate a large number of self-supervised representations, ranking them by interpretability.
  arXiv  Detail & Related papers  (2022-09-07T16:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.