Evolving language compositionality in a frequency-structured meaning space

• URL: http://arxiv.org/abs/2607.29642v1
• Date: Fri, 31 Jul 2026 17:18:57 GMT
• Title: Evolving language compositionality in a frequency-structured meaning space
• Abstract summary: We show how changing the frequency of different meanings affects the character of its compositionality.<n>Results suggest that frequency can shape emergent linguistic structure only when the frequency distribution is defined over form-meaning units.
• Score: 1.5749416770494706
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The iterated learning model was introduced to investigate language evolution: the way in which the characteristic properties of human languages have been shaped, at least partly, by repeated transmission from one language user to another. The key finding is that language compositionality can arise spontaneously as a consequence of language being passed repeatedly through a language learning bottleneck. Here we explore how changing the frequency of different meanings, so that some meanings occur much more frequently than others, affects the character of its compositionality. We find that, as observed in natural languages, high-frequency meanings can escape the pressure to conform to the grammar that characterizes lower-frequency meanings. However, when the frequency structure is instead imposed on parts rather than on whole meaning vectors, the language fails to transmit across generations. This occurs despite the fact that the most frequent elements are reliably learned. These results suggest that frequency can shape emergent linguistic structure only when the frequency distribution is defined over form-meaning units that learners can acquire holistically. When frequency is instead distributed over smaller units, it fails to support the relational structure required for compositional generalisation, thereby preventing stable language transmission.

Related papers

• Image, Word and Thought: A More Challenging Language Task for the Iterated Learning Model [1.7205106391379026]
  Iterated learning model simulates transmission of language from generation to generation.<n>Agents in this model are able to learn and transmit a language that is expressive.
  arXiv  Detail & Related papers  (2026-01-06T10:53:00Z)
• The Harmonic Structure of Information Contours [54.38365999922221]
  We find consistent evidence of periodic patterns in information rate in texts in English, Spanish, German, Dutch, Basque, and Brazilian Portuguese.<n>Many dominant frequencies align with discourse structure, suggesting these oscillations reflect meaningful linguistic organization.
  arXiv  Detail & Related papers  (2025-06-04T12:56:30Z)
• Linguistic Structure from a Bottleneck on Sequential Information Processing [5.850665541267672]
  We show that natural-language-like systematicity arises in codes that are constrained by predictive information. We show that human languages are structured to have low predictive information at the levels of phonology, morphology, syntax, and semantics.
  arXiv  Detail & Related papers  (2024-05-20T15:25:18Z)
• The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments [57.273662221547056]
  In this study, we investigate an unintuitive novel driver of cross-lingual generalisation: language imbalance. We observe that the existence of a predominant language during training boosts the performance of less frequent languages. As we extend our analysis to real languages, we find that infrequent languages still benefit from frequent ones, yet whether language imbalance causes cross-lingual generalisation there is not conclusive.
  arXiv  Detail & Related papers  (2024-04-11T17:58:05Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models [91.3755431537592]
  Representing compositional and non-compositional phrases is critical for language understanding. We first formulate a problem of predicting the LM-internal representations of longer phrases given those of their constituents. While we would expect the predictive accuracy to correlate with human judgments of semantic compositionality, we find this is largely not the case.
  arXiv  Detail & Related papers  (2022-10-07T14:21:30Z)
• Crosslinguistic word order variation reflects evolutionary pressures of dependency and information locality [4.869029215261254]
  About 40% of the world's languages have subject-verb-object order, and about 40% have subject-object-verb order. We show that variation in word order reflects different ways of balancing competing pressures of dependency locality and information locality. Our findings suggest that syntactic structure and usage across languages co-adapt to support efficient communication under limited cognitive resources.
  arXiv  Detail & Related papers  (2022-06-09T02:56:53Z)
• Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang [18.609276255676175]
  We study slang, an informal language that is typically restricted to a specific group or social setting. We analyze the semantic change and frequency shift of slang words and compare them to those of standard, nonslang words. We show that slang words undergo less semantic change but tend to have larger frequency shifts over time.
  arXiv  Detail & Related papers  (2022-03-09T11:34:43Z)
• Linguistic Typology Features from Text: Inferring the Sparse Features of World Atlas of Language Structures [73.06435180872293]
  We construct a recurrent neural network predictor based on byte embeddings and convolutional layers. We show that some features from various linguistic types can be predicted reliably.
  arXiv  Detail & Related papers  (2020-04-30T21:00:53Z)
• Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods [51.34667808471513]
  We investigate the importance of two factors, semantic sparsity and frequency growth rates of semantic neighbors, formalized in the distributional semantics paradigm. We show that both factors are predictive word emergence although we find more support for the latter hypothesis.
  arXiv  Detail & Related papers  (2020-01-21T19:09:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.