When Models Examine Themselves: Vocabulary-Activation Correspondence in Self-Referential Processing

• URL: http://arxiv.org/abs/2602.11358v2
• Date: Wed, 18 Feb 2026 12:06:38 GMT
• Title: When Models Examine Themselves: Vocabulary-Activation Correspondence in Self-Referential Processing
• Abstract summary: We show that self-referential vocabulary tracks concurrent activation dynamics.<n>We identify a direction in activation space that distinguishes self-referential from descriptive processing.<n>Findings indicate that self-report in transformer models can, under appropriate conditions, reliably track internal computational states.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models produce rich introspective language when prompted for self-examination, but whether this language reflects internal computation or sophisticated confabulation has remained unclear. We show that self-referential vocabulary tracks concurrent activation dynamics, and that this correspondence is specific to self-referential processing. We introduce the Pull Methodology, a protocol that elicits extended self-examination through format engineering, and use it to identify a direction in activation space that distinguishes self-referential from descriptive processing in Llama 3.1. The direction is orthogonal to the known refusal direction, localised at 6.25% of model depth, and causally influences introspective output when used for steering. When models produce "loop" vocabulary, their activations exhibit higher autocorrelation (r = 0.44, p = 0.002); when they produce "shimmer" vocabulary under steering, activation variability increases (r = 0.36, p = 0.002). Critically, the same vocabulary in non-self-referential contexts shows no activation correspondence despite nine-fold higher frequency. Qwen 2.5-32B, with no shared training, independently develops different introspective vocabulary tracking different activation metrics, all absent in descriptive controls. The findings indicate that self-report in transformer models can, under appropriate conditions, reliably track internal computational states.

Related papers

• Evaluation Awareness in Language Models: Representation, Verbalization, and Control [5.471648649900293]
  We show that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter.<n>These results show the need for evaluations to account for the disjunction between what models represent internally, what they represent, and their steering.
  arXiv  Detail & Related papers  (2026-08-22T04:16:22Z)
• Causal Interventions on Continuous Variables: A Case Study on Verb Bias in Steering Vectors for In-Context Learning [7.320379735876713]
  We introduce a method for causal intervention on continuous variables.<n>We apply this method to a continuous feature that is well-studied in psycholinguistics, namely verb bias.<n>We show that verb bias is causally represented in steering vectors extracted from large language models.
  arXiv  Detail & Related papers  (2026-05-28T14:09:18Z)
• From Lemmas to Dependencies: What Signals Drive Light Verbs Classification? [0.0]
  Light verb constructions (LVCs) are a challenging class of verbal multiword expressions, especially in Turkish.<n>This paper asks what signals drive LVC classification by systematically restricting model inputs.
  arXiv  Detail & Related papers  (2026-02-04T01:41:11Z)
• LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilization [9.410181019585822]
  We operationalize interpretability methods to ascertain whether we can predict the correctness of model outputs.<n>We consider correct, incorrect, and irrelevant context and introduce metrics to distinguish amongst them.<n>Our model-internals-based metric significantly outperforms prompting baselines at distinguishing between correct and incorrect context.
  arXiv  Detail & Related papers  (2025-10-05T03:14:05Z)
• BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models [55.2480439325792]
  We propose methodology for automated comparison of language models that uses performance-aware contextual embeddings to find fine-grained features of text where one LM outperforms another.<n>Our method, which we name BehaviorBox, extracts coherent features that demonstrate differences with respect to the ease of generation between two LMs.<n>We apply BehaviorBox to compare models that vary in size, model family, and post-training, and enumerate insights into specific contexts that illustrate meaningful differences in performance which cannot be found by measures such as corpus-level perplexity alone.
  arXiv  Detail & Related papers  (2025-06-02T19:44:06Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [102.1527101235251]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios.<n>By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors.<n>LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• LatentQA: Teaching LLMs to Decode Activations Into Natural Language [72.87064562349742]
  We introduce LatentQA, the task of answering open-ended questions about model activations in natural language.<n>We propose Latent Interpretation Tuning (LIT), which finetunes a decoder LLM on a dataset of activations and associated question-answer pairs.<n>Our decoder also specifies a differentiable loss that we use to control models, such as debiasing models on stereotyped sentences and controlling the sentiment of generations.
  arXiv  Detail & Related papers  (2024-12-11T18:59:33Z)
• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Holmes: A Benchmark to Assess the Linguistic Competence of Language Models [59.627729608055006]
  We introduce Holmes, a new benchmark designed to assess language models (LMs) linguistic competence. We use computation-based probing to examine LMs' internal representations regarding distinct linguistic phenomena. As a result, we meet recent calls to disentangle LMs' linguistic competence from other cognitive abilities.
  arXiv  Detail & Related papers  (2024-04-29T17:58:36Z)
• Mapping of attention mechanisms to a generalized Potts model [50.91742043564049]
  We show that training a neural network is exactly equivalent to solving the inverse Potts problem by the so-called pseudo-likelihood method. We also compute the generalization error of self-attention in a model scenario analytically using the replica method.
  arXiv  Detail & Related papers  (2023-04-14T16:32:56Z)
• Discovering Latent Knowledge in Language Models Without Supervision [72.95136739040676]
  Existing techniques for training language models can be misaligned with the truth. We propose directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. We show that despite using no supervision and no model outputs, our method can recover diverse knowledge represented in large language models.
  arXiv  Detail & Related papers  (2022-12-07T18:17:56Z)
• Extracting Latent Steering Vectors from Pretrained Language Models [14.77762401765532]
  We show that latent vectors can be extracted directly from language model decoders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly perfectly. We find that distances between steering vectors reflect sentence similarity when evaluated on a textual similarity benchmark.
  arXiv  Detail & Related papers  (2022-05-10T19:04:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.