Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens

• URL: http://arxiv.org/abs/2609.01936v1
• Date: Tue, 01 Sep 2026 23:01:56 GMT
• Title: Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
• Abstract summary: A lens reading reflects both the hidden state and the readout used to decode it.<n>We introduce Sparse Readout Prism (SRP), which decomposes the readout using only its weights.<n>SRP reveals readout features as a new unit of analysis for lens readings.
• Score: 11.948805377284694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A language model's prediction of its next token develops across layers, and lens methods track this process by decoding intermediate hidden states into tokens. But a lens reading reflects both the hidden state and the readout (the unembedding matrix) used to decode it. Many lenses are fit on a corpus, and we show that two lenses differing only in their fitting corpus can report different tokens for the same hidden states. We call this dependence corpus conditionality. To examine readout structure independently of the fitting corpus, we introduce Sparse Readout Prism (SRP), which decomposes the readout using only its weights and expresses any token logit or logit difference as a sum of contributions from sparse readout features. This reveals readout features as a new unit of analysis for lens readings, exposing structure that token identities can obscure and enabling comparisons across tokens, contexts, layers, and lenses. Replacing the original readout with SRP's sparse approximation reconstructs 8.9-17.3 percentage points more of the tested logit differences than the strongest of six baselines built on geometric relations among readout rows. Ablating features shifts logit differences in proportion to their SRP contributions. Although token readings vary with the fitting corpus, the dominant readout feature remains stable. Because SRP uses no corpus in its construction, it provides a control independent of the fitting corpus for lens analyses.

Related papers

• Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses [0.0]
  We use a training-free tracer that reads at every position, sub-layer, and head average.<n>It traces a compound- noun->attention chain in Phi-2, confirmed there by activation patching.<n>It reads what retrieval surfaces for real versus fictional entities, and reads metaphor as a set of domain-to-domain mappings.
  arXiv  Detail & Related papers  (2026-09-09T09:01:09Z)
• Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora? [43.96699318814863]
  We first show that BPE tokenizers trained on different corpora share stable token ID-- distributions, motivating distribution transfer from known corpora to a target tokenizer trained on hidden corpora.<n>We then propose Quantile-Guided Density Estimation (QGDE), which approximates this distribution with multiple quantile trends and uses local density weighting to produce token-level estimates.
  arXiv  Detail & Related papers  (2026-08-11T09:14:25Z)
• The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric [78.74729751232852]
  We introduce a large-scale dataset of human similarity judgments over image triplets.<n>We fine-tune a vision-language model to produce our Text-Prompted Image Perceptual Similarity metric.<n>TPIPS aligns more closely with human perception and generalizes reliably beyond the training distribution.
  arXiv  Detail & Related papers  (2026-07-20T17:59:51Z)
• PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentation [58.1914505657064]
  This paper proposes a simple yet effective parallel cost aggregation (PCA-Seg) paradigm to alleviate the challenge of knowledge interference between class-level semantics and spatial context.<n>Experiments on eight benchmarks show that each parallel block in PCA-Seg adds merely 0.35M parameters while achieving state-of-the-art OSPS performance.
  arXiv  Detail & Related papers  (2026-03-18T09:26:43Z)
• Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruning [78.75062483648243]
  Vision-language models (VLMs) often generate massive visual tokens that greatly increase inference latency and memory footprint.<n>We propose Focus-Scan-Refine (FSR), a human-inspired, plug-and-play pruning framework that mimics how humans answer visual questions.<n>FSR consistently improves the accuracy-efficiency trade-off over existing state-of-the-art pruning methods.
  arXiv  Detail & Related papers  (2026-02-05T16:02:48Z)
• Mechanistic Decomposition of Sentence Representations [3.9146761527401432]
  Sentence embeddings are central to modern NLP and AI systems, but little is known about their internal structure.<n>We propose a new method to mechanistically decompose sentence embeddings into interpretable components.<n>We analyze how pooling compresses these features into sentence representations, and assess the latent features that reside in a sentence embedding.
  arXiv  Detail & Related papers  (2025-06-04T18:42:57Z)
• From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance [14.30327576545802]
  We introduce a framework that reveals what latent components activate for, how they align with expected semantics, and how important they are to predictions.<n>Our method uncovers hundreds of surprising components linked to polysemous words, compound nouns, visual typography and dataset artifacts.
  arXiv  Detail & Related papers  (2025-05-26T17:08:02Z)
• QUDsim: Quantifying Discourse Similarities in LLM-Generated Text [70.22275200293964]
  We introduce an abstraction based on linguistic theories in Questions Under Discussion (QUD) and question semantics to help quantify differences in discourse progression.<n>We then use this framework to build $textbfQUDsim$, a similarity metric that can detect discursive parallels between documents.<n>Using QUDsim, we find that LLMs often reuse discourse structures (more so than humans) across samples, even when content differs.
  arXiv  Detail & Related papers  (2025-04-12T23:46:09Z)
• Subobject-level Image Tokenization [60.80949852899857]
  Patch-based image tokenization ignores the morphology of the visual world.<n>Inspired by subword tokenization, we introduce subobject-level adaptive token segmentation.<n>We show that subobject tokenization enables faster convergence and better generalization while using fewer visual tokens.
  arXiv  Detail & Related papers  (2024-02-22T06:47:44Z)
• Siamese based Neural Network for Offline Writer Identification on word level data [7.747239584541488]
  We propose a novel scheme to identify the author of a document based on the input word image. Our method is text independent and does not impose any constraint on the size of the input image under examination.
  arXiv  Detail & Related papers  (2022-11-17T10:01:46Z)
• Explaining Latent Representations with a Corpus of Examples [72.50996504722293]
  We propose SimplEx: a user-centred method that provides example-based explanations with reference to a freely selected set of examples. SimplEx uses the corpus to improve the user's understanding of the latent space with post-hoc explanations. We show that SimplEx empowers the user by highlighting relevant patterns in the corpus that explain model representations.
  arXiv  Detail & Related papers  (2021-10-28T17:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.