Planting a Latent Variable in Natural-Looking Text: a More Realistic Test of Belief States in LLMs and Their Link to Concept Geometry

• URL: http://arxiv.org/abs/2608.26887v1
• Date: Thu, 27 Aug 2026 09:45:00 GMT
• Title: Planting a Latent Variable in Natural-Looking Text: a More Realistic Test of Belief States in LLMs and Their Link to Concept Geometry
• Abstract summary: We plant a controllable latent variable inside natural-looking text.<n>A small transformer model trained on this corpus does indeed track the Bayesian posterior belief about our planted latent variable.
• Score: 51.99454801210509
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs are thought to track "belief states," i.e., running probability distributions over the latent variables that govern language (Shai et al., 2024; Sarfati et al., 2026), but so far this has only been comprehensively demonstrated on toy synthetic data and in a few isolated case studies. It has also never been empirically connected to the geometry of LLM features (the concepts interpretability finds in model activations). In this work, we plant a controllable latent variable inside natural-looking text. An LLM teacher writes ordinary text while we "subliminally" steer it along one of K = 8 unrelated sparse autoencoder directions at each token, with the active directions following a ring-shaped Markov chain. A small transformer model trained on this corpus does indeed track the Bayesian posterior belief about our planted latent variable. Moreover, it also arranges the 8 states themselves on a ring, in the exact order of the Markov chain, which is supporting evidence that a concept's geometry can be formed by the statistical dynamics of the latent variable behind it.

Related papers

• Linguistic Holonomy and Statistical Watermarks: Inner Geometry of Meaning-Preserving Transformations [0.0]
  We show that the endpoint of transformations which move a text while leaving its content in place is the wrong statistic.<n>We adapt the formalism of linguistic loops to show that the invariant of a chain of meaning-preserving transformations factorises canonically into an endpoint part and a holonomy in the stabiliser of the initial state.<n>On the side of the detector we prove an exact identity: the residual statistic is proportional to the number of positions whose seeding window survived intact.
  arXiv  Detail & Related papers  (2026-08-19T18:36:51Z)
• Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought [72.45900226435289]
  Large language models (LLMs) can generate long Chain-of-Thought (CoT) at test time, enabling them to solve complex tasks.<n>We measure the step-wise causal influence of each reasoning step on the model's final prediction with a proposed True Thinking Score (TTS)<n>We identify a TrueThinking direction in the latent space of LLMs, which can force the model to perform or disregard certain CoT steps.
  arXiv  Detail & Related papers  (2025-10-28T20:14:02Z)
• (How) Do Language Models Track State? [52.03388573068501]
  Transformer language models (LMs) exhibit behaviors that seem to require tracking the unobserved state of an evolving world.<n>We study state tracking in LMs trained or fine-tuned to compose permutations.<n>We show that LMs consistently learn one of two state tracking mechanisms for this task.
  arXiv  Detail & Related papers  (2025-03-04T18:31:02Z)
• Unsupervised Representation Learning from Sparse Transformation Analysis [79.94858534887801]
  We propose to learn representations from sequence data by factorizing the transformations of the latent variables into sparse components. Input data are first encoded as distributions of latent activations and subsequently transformed using a probability flow model.
  arXiv  Detail & Related papers  (2024-10-07T23:53:25Z)
• A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation [72.93327642336078]
  We propose Belief Tree Propagation (BTProp), a probabilistic framework for hallucination detection.<n>BTProp introduces a belief tree of logically related statements by decomposing a parent statement into child statements.<n>Our method improves baselines by 3%-9% (evaluated by AUROC and AUC-PR) on multiple hallucination detection benchmarks.
  arXiv  Detail & Related papers  (2024-06-11T05:21:37Z)
• What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages [78.1866280652834]
  Large language models (LM) are distributions over strings.<n>We investigate the learnability of regular LMs (RLMs) by RNN and Transformer LMs.<n>We find that the complexity of the RLM rank is strong and significant predictors of learnability for both RNNs and Transformers.
  arXiv  Detail & Related papers  (2024-06-06T17:34:24Z)
• The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains [28.41876902994335]
  We introduce a simple Markov Chain sequence modeling task to study how this in-context learning (ICL) capability emerges. Transformers trained on this task form emphstatistical induction heads which compute accurate next-token probabilities. We show how successful learning results from the interaction between the transformer's layers, and uncover evidence that the presence of the simpler unigram solution may delay formation of the final bigram solution.
  arXiv  Detail & Related papers  (2024-02-16T18:28:36Z)
• Deep de Finetti: Recovering Topic Distributions from Large Language Models [10.151434138893034]
  Large language models (LLMs) can produce long, coherent passages of text. LLMs must represent the latent structure that characterizes a document. We investigate a complementary aspect, namely the document's topic structure.
  arXiv  Detail & Related papers  (2023-12-21T16:44:39Z)
• What's the Magic Word? A Control Theory of LLM Prompting [0.7499722271664144]
  We explore prompt engineering through the lens of control theory. We present empirical results on the controllability of a panel of LLMs. Short prompt sequences can dramatically alter the likelihood of specific outputs.
  arXiv  Detail & Related papers  (2023-10-02T22:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.