Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs

• URL: http://arxiv.org/abs/2602.10352v1
• Date: Tue, 10 Feb 2026 22:50:02 GMT
• Title: Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs
• Abstract summary: Self-interpretation methods prompt language models to describe their own internal states.<n>We show that training lightweight adapters on interpretability artifacts, while keeping the LM entirely frozen, yields reliable self-interpretation.
• Score: 1.028001761831691
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-interpretation methods prompt language models to describe their own internal states, but remain unreliable due to hyperparameter sensitivity. We show that training lightweight adapters on interpretability artifacts, while keeping the LM entirely frozen, yields reliable self-interpretation across tasks and model families. A scalar affine adapter with just $d_\text{model}+1$ parameters suffices: trained adapters generate sparse autoencoder feature labels that outperform the training labels themselves (71% vs 63% generation scoring at 70B scale), identify topics with 94% recall@1 versus 1% for untrained baselines, and decode bridge entities in multi-hop reasoning that appear in neither prompt nor response, surfacing implicit reasoning without chain-of-thought. The learned bias vector alone accounts for 85% of improvement, and simpler adapters generalize better than more expressive alternatives. Controlling for model knowledge via prompted descriptions, we find self-interpretation gains outpace capability gains from 7B to 72B parameters. Our results demonstrate that self-interpretation improves with scale, without modifying the model being interpreted.

Related papers

• Scaling Inherently Interpretable Language Models [8.157444017363812]
  We make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective.<n>We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask.<n>For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data.
  arXiv  Detail & Related papers  (2026-08-06T04:29:58Z)
• Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance [58.49623467782041]
  ELUDe is a method for improving the interpretability of deep neural networks.<n>It breaks latent representations into clear, inspectable sub-units that behave like interpretable features.<n>It requires no explicit training, no labels, and can be applied to pretrained models.
  arXiv  Detail & Related papers  (2026-05-29T13:39:09Z)
• Probing for Knowledge Attribution in Large Language Models [45.47366023067617]
  Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations.<n> Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights.<n>We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution.
  arXiv  Detail & Related papers  (2026-02-26T09:21:12Z)
• Transcoder Adapters for Reasoning-Model Diffing [28.090312772463253]
  We introduce transcoder computation, a technique for learning an interpretable approximation.<n>We apply transcoder adapters to characterize the differences between Qwen2.5-Math-7B and its reasoning-distilled variant, DeepSeek-R1-Di-Qwen-7B.
  arXiv  Detail & Related papers  (2026-02-24T13:40:28Z)
• Direct Semantic Communication Between Large Language Models via Vector Translation [3.81908263930559]
  Large language models (LLMs) pass messages as plain tokens, discarding most latent semantics.<n>We form a latent bridge via vector translations, which use learned mappings that enable direct semantic exchange.
  arXiv  Detail & Related papers  (2025-11-06T00:43:29Z)
• Regularization Through Reasoning: Systematic Improvements in Language Model Classification via Explanation-Enhanced Fine-Tuning [2.247737938202007]
  We evaluate whether attaching brief explanations to each label during fine-tuning yields better models.<n>We replace human-written explanations with text that is syntactically incoherent yet vocabulary-aligned with the originals.<n>The effect persists across datasets and training seeds, indicating that gains arise less from meaning than from structure.
  arXiv  Detail & Related papers  (2025-11-03T20:25:42Z)
• Verifying Large Language Models' Reasoning Paths via Correlation Matrix Rank [71.09032766271493]
  Large language models (LLMs) are prone to errors and hallucinations.<n>How to check their outputs effectively and efficiently has become a critical problem in their applications.
  arXiv  Detail & Related papers  (2025-10-28T11:01:10Z)
• Do What? Teaching Vision-Language-Action Models to Reject the Impossible [53.40183895299108]
  Vision-Language-Action (VLA) models have demonstrated strong performance on a range of robotic tasks.<n>We propose Instruct-Verify-and-Act (IVA), a framework that detects when an instruction cannot be executed due to a false premise.<n>Our experiments show that IVA improves false premise detection accuracy by 97.56% over baselines.
  arXiv  Detail & Related papers  (2025-08-22T10:54:33Z)
• SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models [88.29990536278167]
  We introduce SPaR, a self-play framework integrating tree-search self-refinement to yield valid and comparable preference pairs.<n>Our experiments show that a LLaMA3-8B model, trained over three iterations guided by SPaR, surpasses GPT-4-Turbo on the IFEval benchmark without losing general capabilities.
  arXiv  Detail & Related papers  (2024-12-16T09:47:43Z)
• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora? [3.780441744500289]
  We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Our findings reveal alarming inconsistencies across all languages and models. If not promptly addressed, simplified inputs can be easily exploited to craft zero-it model-agnostic adversarial attacks with success rates of up to 50%.
  arXiv  Detail & Related papers  (2024-04-10T09:02:33Z)
• Interpretability at Scale: Identifying Causal Mechanisms in Alpaca [62.65877150123775]
  We use Boundless DAS to efficiently search for interpretable causal structure in large language models while they follow instructions. Our findings mark a first step toward faithfully understanding the inner-workings of our ever-growing and most widely deployed language models.
  arXiv  Detail & Related papers  (2023-05-15T17:15:40Z)
• Does entity abstraction help generative Transformers reason? [8.159805544989359]
  We study the utility of incorporating entity type abstractions into pre-trained Transformers. We test these methods on four NLP tasks requiring different forms of logical reasoning.
  arXiv  Detail & Related papers  (2022-01-05T19:00:53Z)
• Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
  Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
  arXiv  Detail & Related papers  (2021-10-04T08:51:36Z)
• Learning to Faithfully Rationalize by Construction [36.572594249534866]
  In many settings it is important to be able to understand why a model made a particular prediction. We propose a simpler variant of this approach that provides faithful explanations by construction. In both automatic and manual evaluations we find that variants of this simple framework yield superior to end-to-end' approaches.
  arXiv  Detail & Related papers  (2020-04-30T21:45:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.