One mechanism for many mental spaces: a shared router over a value slot in language models

• URL: http://arxiv.org/abs/2607.10248v1
• Date: Sat, 11 Jul 2026 10:30:36 GMT
• Title: One mechanism for many mental spaces: a shared router over a value slot in language models
• Abstract summary: Language builds discourse contexts other than the actual.<n>Formal semantics keeps these contexts apart because their logics differ.<n>We ask which of these a transformer language model implements, and find a mechanistic version of Fauconnier's unification.
• Score: 6.0397270384735355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language builds discourse contexts other than the actual: a painting, a belief, a memory, a hypothetical. Each is a mental space in which the same entity can take a different value, as when a flower is red in reality but purple in a portrait. Formal semantics keeps these contexts apart because their logics differ (modal, temporal, doxastic, depictive); Fauconnier's mental-space theory treats them as one space-building operation. We ask which of these a transformer language model implements, and find a mechanistic version of Fauconnier's unification. The model uses one router/slot format across the inventory: a reusable value slot stores attributed content, and a causally manipulable router (the space index) selects which space is read. A subspace trained with Distributed Alignment Search to control one space type, counterfactual, belief, fictional, or temporal, also controls the others, well above a random floor, on three model families; belief, which formal semantics marks as a distinct case, is not specially separated. The router is low-rank, composes additively with entity identity, and acts through a few late-layer heads. Two further results show the mechanism drives inference and composes: a subspace trained on a rule-derived conclusion flips what the model infers while dissociating from what it reports, and composing space-builders mints a fresh router over the shared slot. This paper establishes the cross-type generality. A companion paper develops belief in depth, because of its special status in philosophy, psychology, and linguistics (epistemology, theory of mind, and propositional attitude reports).

Related papers

• Belief-reality separation lives in routing over a shared value slot in language models [6.0397270384735355]
  Capable language models hold what a character believes apart from what is true.<n>We show it rests on two separable mechanisms at two positions.
  arXiv  Detail & Related papers  (2026-07-11T10:39:03Z)
• A Mechanistic Understanding of Pronoun Fidelity in LLMs [21.554337243825884]
  We study whether three mechanisms -- group entity binding (G), recency bias (R), and stereotypical bias (S) -- are causally implemented across several SOTA language models.<n>No single mechanism fully explains model behaviour, but a combination of the three consistently accounts for 99.5%1-9.<n>In sum, pronoun fidelity arises from competition between simultaneously active causal subspaces.
  arXiv  Detail & Related papers  (2026-06-15T08:45:53Z)
• OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling [49.9159527733006]
  Theory of Mind (ToM) is commonly evaluated in large language models (LLMs) using end-point question answering.<n>This paradigm obscures whether the model actually constructs the underlying mental-state representations required for robust reasoning.<n>We introduce OmniToM, a benchmark that directly evaluates these representations by requiring explicit modeling of belief structures for all relevant actors within a narrative.
  arXiv  Detail & Related papers  (2026-05-25T20:45:08Z)
• Interpretation as Linear Transformation: A Cognitive-Geometric Model of Belief and Meaning [0.0]
  I show how belief distortion, motivational drift, counterfactual evaluation, and the limits of mutual understanding arise from purely algebraic constraints.<n>I argue that this cognitive-geometric perspective clarifies the boundaries of influence in both human and artificial systems.
  arXiv  Detail & Related papers  (2025-12-10T17:13:01Z)
• Emergence of Linear Truth Encodings in Language Models [64.86571541830598]
  Large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear.<n>We introduce a transparent, one-layer transformer toy model that reproduces such truth subspaces end-to-end.<n>We study one simple setting in which truth encoding can emerge, encouraging the model to learn this distinction in order to lower the LM loss on future tokens.
  arXiv  Detail & Related papers  (2025-10-17T16:30:07Z)
• A Geometric Notion of Causal Probing [85.49839090913515]
  The linear subspace hypothesis states that, in a language model's representation space, all information about a concept such as verbal number is encoded in a linear subspace.<n>We give a set of intrinsic criteria which characterize an ideal linear concept subspace.<n>We find that, for at least one concept across two languages models, the concept subspace can be used to manipulate the concept value of the generated word with precision.
  arXiv  Detail & Related papers  (2023-07-27T17:57:57Z)
• The Hidden Language of Diffusion Models [70.03691458189604]
  We present Conceptor, a novel method to interpret the internal representation of a textual concept by a diffusion model. We find surprising visual connections between concepts, that transcend their textual semantics. We additionally discover concepts that rely on mixtures of exemplars, biases, renowned artistic styles, or a simultaneous fusion of multiple meanings.
  arXiv  Detail & Related papers  (2023-06-01T17:57:08Z)
• Latent Topology Induction for Understanding Contextualized Representations [84.7918739062235]
  We study the representation space of contextualized embeddings and gain insight into the hidden topology of large language models. We show there exists a network of latent states that summarize linguistic properties of contextualized representations.
  arXiv  Detail & Related papers  (2022-06-03T11:22:48Z)
• Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks [73.94290462239061]
  We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation. By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
  arXiv  Detail & Related papers  (2022-01-14T14:54:58Z)
• Talking Space: inference from spatial linguistic meanings [0.0]
  This paper concerns the intersection of natural language and the physical space around us in which we live. We propose a mechanism for how space and linguistic structure can be made to interact in a matching compositional fashion.
  arXiv  Detail & Related papers  (2021-09-14T09:53:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.