Word Class Representations Spontaneously Emerge from Successor Representations Trained on Natural Language

• URL: http://arxiv.org/abs/2605.24585v1
• Date: Sat, 23 May 2026 13:55:09 GMT
• Title: Word Class Representations Spontaneously Emerge from Successor Representations Trained on Natural Language
• Abstract summary: Language models are typically trained to predict the next token in a sequence.<n>Here, we explore an alternative predictive principle from reinforcement learning: Successor Representations.<n>We train neural networks to predict future word distributions across multiple temporal horizons.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Language models are typically trained to predict the next token in a sequence. Here, we explore an alternative predictive principle from reinforcement learning: Successor Representations (SRs), which model the expected discounted distribution of future states rather than the immediate next state. We transfer this framework to natural language and train neural networks to predict future word distributions across multiple temporal horizons, thereby learning representations of long-range transition structure. We train a deep residual neural network on WikiText-103 (103 million tokens; 20,000-word vocabulary) and optimize successor representations as probability distributions using KL divergence. Without explicit linguistic supervision, structured language representations emerge spontaneously. After training, the learned space develops a clear geometric organization with respect to part-of-speech (POS) categories: nouns, verbs, and adjectives become separable and recoverable through unsupervised clustering. This organization depends systematically on predictive horizon, with short horizons producing the strongest syntactic structure and longer horizons increasingly integrating broader contextual and semantic information. At finer resolutions, additional interpretable lexical substructure emerges, revealing coherent subclasses within major word categories. These findings suggest that syntactic categories need not be explicitly encoded but may arise as a consequence of predictive sequence learning. To our knowledge, this work provides the first systematic application of successor representations to natural language and establishes a conceptual bridge between reinforcement learning, linguistics, and cognitive neuroscience.

Related papers

• Formal Properties of Language as Constraints on Neural Dynamics [0.20305676256390937]
  Current research annotates stimuli with linguistic variables and tests which saturate electrodes, voxels, or language-model layers predict neural activity.<n>Here, we show that algebraic properties of language specify invariants that mechanisms must preserve.<n>We propose a novel neural binding operation we term 'Meld': two constituent populations converge through shared synapses, integrate sublinearly, and Merge.
  arXiv  Detail & Related papers  (2026-09-13T08:41:56Z)
• Conceptual Networks for Cross-Linguistic Idiomatic Expressions: A Feature-Based Graph Approach [0.0]
  We present an interpretable network-based framework for representing idiomatic and figurative meaning across eight typologically diverse languages.<n>Each expression is annotated with binary conceptual features derived from cognitive-linguistic theory, and pairwise Jaccard similarities define a weighted graph.<n>Community detection reveals that idioms cluster by conceptual schema rather than by language, producing a structure consistent with cognitive-linguistic predictions.
  arXiv  Detail & Related papers  (2026-07-10T16:25:51Z)
• Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretability [31.30541946703775]
  Translating internal representations and computations of models into concepts that humans can understand is a key goal of interpretability.<n>Recent dictionary learning methods such as Sparse Autoencoders provide a promising route to discover human-interpretable features.<n>But they exhibit a bias towards shallow, token-specific, or noisy features, such as "the phrase 'The' at the start of sentences"
  arXiv  Detail & Related papers  (2025-10-30T17:59:30Z)
• Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations [34.88156871518115]
  Next-token prediction (NTP) optimization leads language models to extract and organize semantic structure from text.<n>We demonstrate that concepts corresponding to larger singular values are learned earlier during training, yielding a natural semantic hierarchy.<n>This insight motivates orthant-based clustering, a method that combines concept signs to identify interpretable semantic categories.
  arXiv  Detail & Related papers  (2025-05-13T08:46:04Z)
• Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures [49.19753720526998]
  We derive theoretical scaling laws for neural network performance on synthetic datasets.<n>We validate that convolutional networks, whose structure aligns with that of the generative process through locality and weight sharing, enjoy a faster scaling of performance.<n>This finding clarifies the architectural biases underlying neural scaling laws and highlights how representation learning is shaped by the interaction between model architecture and the statistical properties of data.
  arXiv  Detail & Related papers  (2025-05-11T17:44:14Z)
• Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning [29.745218855471787]
  Tokenization is a necessary component within the current architecture of many language mod-els.<n>We argue that tokenization is necessary for reasonably human-like language performance.<n>We discuss implications for architectural choices, meaning construction, the primacy of language for thought.
  arXiv  Detail & Related papers  (2024-12-14T18:18:52Z)
• Collapsed Language Models Promote Fairness [88.48232731113306]
  We find that debiased language models exhibit collapsed alignment between token representations and word embeddings.<n>We design a principled fine-tuning method that can effectively improve fairness in a wide range of debiasing methods.
  arXiv  Detail & Related papers  (2024-10-06T13:09:48Z)
• Word class representations spontaneously emerge in a deep neural network trained on next word prediction [7.240611820374677]
  How do humans learn language, and can the first language be learned at all? These fundamental questions are still hotly debated. In particular, we train an artificial deep neural network on predicting the next word. We find that the internal representations of nine-word input sequences cluster according to the word class of the tenth word to be predicted as output.
  arXiv  Detail & Related papers  (2023-02-15T11:02:50Z)
• Latent Topology Induction for Understanding Contextualized Representations [84.7918739062235]
  We study the representation space of contextualized embeddings and gain insight into the hidden topology of large language models. We show there exists a network of latent states that summarize linguistic properties of contextualized representations.
  arXiv  Detail & Related papers  (2022-06-03T11:22:48Z)
• Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining [8.254139827478355]
  We develop a framework that exploits the cognitive mechanisms of chaining and multimodal knowledge to predict compositional expressions through time. We present the syntactic frame extension model (SFEM) that draws on the theory of chaining and knowledge from "percept", "concept", and "language" We show that multimodal SFEM predicts newly emerged verb syntax and arguments substantially better than competing models using purely linguistic or unimodal knowledge.
  arXiv  Detail & Related papers  (2021-09-10T03:42:07Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• SLM: Learning a Discourse Language Representation with Sentence Unshuffling [53.42814722621715]
  We introduce Sentence-level Language Modeling, a new pre-training objective for learning a discourse language representation. We show that this feature of our model improves the performance of the original BERT by large margins.
  arXiv  Detail & Related papers  (2020-10-30T13:33:41Z)
• APo-VAE: Text Generation in Hyperbolic Space [116.11974607497986]
  In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations. An Adrial Poincare Variversaational Autoencoder (APo-VAE) is presented, where both the prior and variational posterior of latent variables are defined over a Poincare ball via wrapped normal distributions. Experiments in language modeling and dialog-response generation tasks demonstrate the winning effectiveness of the proposed APo-VAE model.
  arXiv  Detail & Related papers  (2020-04-30T19:05:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.