Metaphor Tracer: A Theory-Informed Analysis of Hidden States

• URL: http://arxiv.org/abs/2607.28434v1
• Date: Thu, 30 Jul 2026 16:12:13 GMT
• Title: Metaphor Tracer: A Theory-Informed Analysis of Hidden States
• Abstract summary: From one forward pass, without training, we score every token position on two properties.<n>The *aggregator* measures whether the position consolidates the whole text into a stable configuration.<n>The *differentiator*, whether other tokens are transiently carried into its subspace as the model reads.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: What do a language model's hidden states say about the organization of a single text? From one forward pass, without training, we score every token position on two properties. The *aggregator* measures whether the position consolidates the whole text into a stable configuration. The *differentiator*, whether other tokens are transiently carried into its subspace as the model reads: metaphor in its root sense, transport. Constants were frozen on one discovery text; every other is confirmatory. The aggregator is not, in the classic sense, an information measure, nor a measure of salience. Across three unrelated models, as a signifier repeats, its surprisal and its attention drain while its aggregator score holds: the channel marks a token's place in the text. That this tracks a reading rests on independent ground truth: an engineered register the aggregator follows across its boundaries (6/6 cells), and a psychoanalyst's marking of clinical transcripts, fixed before the instrument existed, in 34/36 cells, with a graded increment above lexical controls and dissociations no type-level measure reproduces. A transfer test gives the result its shape: the model whose token structure travels with lexical type reads the singular discourse worst, and in a matched base/instruct pair tuning raises fidelity without moving type-transfer. Structural value is a property of a token's place in *this* text, not of its vector alone: a relational rather than essentialist reading of hidden states, operationalizing theory that predated the instrument.

Related papers

• Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses [0.0]
  We use a training-free tracer that reads at every position, sub-layer, and head average.<n>It traces a compound- noun->attention chain in Phi-2, confirmed there by activation patching.<n>It reads what retrieval surfaces for real versus fictional entities, and reads metaphor as a set of domain-to-domain mappings.
  arXiv  Detail & Related papers  (2026-09-09T09:01:09Z)
• Planting a Latent Variable in Natural-Looking Text: a More Realistic Test of Belief States in LLMs and Their Link to Concept Geometry [51.99454801210509]
  We plant a controllable latent variable inside natural-looking text.<n>A small transformer model trained on this corpus does indeed track the Bayesian posterior belief about our planted latent variable.
  arXiv  Detail & Related papers  (2026-08-27T09:45:00Z)
• Linguistic Holonomy and Statistical Watermarks: Inner Geometry of Meaning-Preserving Transformations [0.0]
  We show that the endpoint of transformations which move a text while leaving its content in place is the wrong statistic.<n>We adapt the formalism of linguistic loops to show that the invariant of a chain of meaning-preserving transformations factorises canonically into an endpoint part and a holonomy in the stabiliser of the initial state.<n>On the side of the detector we prove an exact identity: the residual statistic is proportional to the number of positions whose seeding window survived intact.
  arXiv  Detail & Related papers  (2026-08-19T18:36:51Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• The Information Shadow: Measuring Structural Limits on What Language Models Can Learn [0.0]
  Information shadow: phenomena that a text-trained learner cannot acquire regardless of scale.<n>For Type I, Language Compression Residuals compare a text learner, which sees only a lossy text-like encoding of the signal, against a full-signal learner, which sees the underlying signal directly.<n>For Type II, the Counterfactual Distinction Test trains models on data exactly consistent with two incompatible rules.
  arXiv  Detail & Related papers  (2026-07-17T06:04:07Z)
• Legible-by-Construction: Attention and End-to-End Transformers [0.2663471820643486]
  A transformer's feed-forward layer can be rebuilt from explicit fuzzy set operations.<n> hidden units read as named logical operators at no cost to language-model quality.<n>Across five specialized-attention designs at 125M parameters, 44 to 62 percent of value channels become legible.
  arXiv  Detail & Related papers  (2026-07-05T14:08:59Z)
• Can Machines Really See Objects in Images? A Study Based on Syntactic Distance and Visual Self-Referential Instances [53.65561607407989]
  We ask whether a model can still classify correctly when such local cues provide no stable basis for distinction.<n>Experiments on ResNets and Vision Transformers reveal a consistent phase-transition phenomenon.<n>Results suggest that general intelligence may require creating new language, not reusing an existing one.
  arXiv  Detail & Related papers  (2026-06-28T14:27:40Z)
• Structure Before Collapse: Transient semantic geometry in next-token prediction [33.942301586178914]
  Neural Collapse predicts that balanced one-hot classification pushes model representations to be equally far from each other.<n>This creates a puzzle: next-token prediction language models are trained predominantly with one-hot labels.<n>We find that semantic geometry emerges early in training, and that representations cluster by shared attributes.
  arXiv  Detail & Related papers  (2026-06-25T08:33:34Z)
• Not All Synthetic Data Is Yours to Learn From [68.13136636413601]
  We show that weak self-training can amplify capabilities already present in the pretrained model.<n>We study this in the minimal setting of prompt-free unconditional self-training.
  arXiv  Detail & Related papers  (2026-05-29T10:34:11Z)
• A Geometric Profile of Semantic Information in Text: Frame-Conditional Uniqueness and a Trade-Off Triangle for Scalar Summaries [0.0]
  We develop a framework that measures semantic content from the structure of a text's sentence embeddings.<n>The recommended rank-normalized configuration passes 25 of 28 ordinal checks as point estimates.<n>A separate variational result connects the breadth coordinate to the log-determinant of a determinantal point process.
  arXiv  Detail & Related papers  (2026-05-27T04:37:26Z)
• Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation [34.50304918655722]
  Transformer representations are known to carry label-level information, but whether they encode semantic operations producing those labels is unclear.<n>We investigate this in Natural Language Inference using controlled premise-hypothesis pairs that differ by a single semantic transformation.
  arXiv  Detail & Related papers  (2026-05-25T07:21:07Z)
• Lexinvariant Language Models [84.2829117441298]
  Token embeddings, a mapping from discrete lexical symbols to continuous vectors, are at the heart of any language model (LM) We study textitlexinvariantlanguage models that are invariant to lexical symbols and therefore do not need fixed token embeddings in practice. We show that a lexinvariant LM can attain perplexity comparable to that of a standard language model, given a sufficiently long context.
  arXiv  Detail & Related papers  (2023-05-24T19:10:46Z)
• How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding [56.222097640468306]
  We provide mechanistic understanding of how transformers learn "semantic structure" We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
  arXiv  Detail & Related papers  (2023-03-07T21:42:17Z)
• Neural Deepfake Detection with Factual Structure of Text [78.30080218908849]
  We propose a graph-based model for deepfake detection of text. Our approach represents the factual structure of a given document as an entity graph. Our model can distinguish the difference in the factual structure between machine-generated text and human-written text.
  arXiv  Detail & Related papers  (2020-10-15T02:35:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.