Language models struggle with compartmentalization

• URL: http://arxiv.org/abs/2605.19284v1
• Date: Tue, 19 May 2026 03:02:46 GMT
• Title: Language models struggle with compartmentalization
• Abstract summary: We show that large language models (LLMs) fail to identify and share statistical strength between distinct presentations of unified concepts.<n>We also demonstrate that synthetic parallel data can fail to improve this despite being easily learned itself.<n>All interventions that we study exhibit a phase transition in which their effectiveness depends on the number of distinct presentations.
• Score: 3.7098038388802252
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the training data used by large language models (LLMs), the same latent concept is often presented in multiple distinct ways: the same facts appear in English and Swahili; many functions can be expressed in both Python and Haskell; we can express propositions in both formal and natural language. We show that LLMs can exhibit compartmentalization, where they fail to identify and share statistical strength between distinct presentations of unified concepts. In the worst case, LLMs simply learn parallel internal representations of each presentation of the concept, saturating model capacity with redundancies and decreasing sample efficiency with the number of such presentations. We also demonstrate that synthetic parallel data can fail to improve this despite being easily learned itself. Under this framework, we find that, for small models, early multilingual learning is nearly entirely compartmentalized. Finally, all interventions that we study exhibit a phase transition in which their effectiveness depends on the number of distinct presentations, suggesting that the language modeling objective may only inconsistently unify representations.

Related papers

• We Hebben Een Serieus Translatie: Modeling Intercomprehension as Probabilistic Inference [10.293916944634804]
  Intercomprehension refers to partial intelligibility of an unfamiliar language by a speaker of a related language.<n>In this work, we extend past work on algorithmic models of noisy-channel inference to model intercomprehension in a Bayesian framework.<n>We conduct a human behavioral experiment, eliciting inferences for utterances in Dutch, Italian, and Ukrainian from speakers of English, Spanish, and Russian, respectively.
  arXiv  Detail & Related papers  (2026-07-13T21:30:05Z)
• Language-Specific Latent Process Hinders Cross-Lingual Performance [38.36668133949413]
  Large language models (LLMs) are capable of cross-lingual transfer, but can produce inconsistent output when prompted with the same queries written in different languages.<n>We measure representation similarity between languages, and apply the logit lens to interpret the implicit steps taken by LLMs to solve multilingual multi-choice reasoning questions.<n>Our analyses reveal LLMs predict inconsistently and are less accurate because they rely on representations that are dissimilar across languages, rather than working in a shared semantic space.
  arXiv  Detail & Related papers  (2025-05-19T14:10:15Z)
• Feasibility with Language Models for Open-World Compositional Zero-Shot Learning [96.6544564242316]
  In Open-World Compositional Zero-Shot Learning, all possible state-object combinations are considered as unseen classes.<n>Our work focuses on using external auxiliary knowledge to determine the feasibility of state-object combinations.
  arXiv  Detail & Related papers  (2025-05-16T12:37:08Z)
• Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning [50.00090601424348]
  Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks. We propose EASE, an Explanation-Aware Soft Ensemble framework to empower in-context learning with LLMs.
  arXiv  Detail & Related papers  (2023-11-13T06:13:38Z)
• Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds [59.71218039095155]
  We evaluate language understanding capacities on simple inference tasks that most humans find trivial. We target (i) grammatically-specified entailments, (ii) premises with evidential adverbs of uncertainty, and (iii) monotonicity entailments. The models exhibit moderate to low performance on these evaluation sets.
  arXiv  Detail & Related papers  (2023-05-24T06:41:09Z)
• Evaluating statistical language models as pragmatic reasoners [39.72348730045737]
  We evaluate the capacity of large language models to infer meanings of pragmatic utterances. We find that LLMs can derive context-grounded, human-like distributions over the interpretations of several complex pragmatic utterances. Results inform the inferential capacity of statistical language models, and their use in pragmatic and semantic parsing applications.
  arXiv  Detail & Related papers  (2023-05-01T18:22:10Z)
• Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning [104.58874584354787]
  In recent years, pre-trained large language models (LLMs) have demonstrated remarkable efficiency in achieving an inference-time few-shot learning capability known as in-context learning. This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models.
  arXiv  Detail & Related papers  (2023-01-27T18:59:01Z)
• On the Compositional Generalization Gap of In-Context Learning [73.09193595292233]
  We look at the gap between the in-distribution (ID) and out-of-distribution (OOD) performance of such models in semantic parsing tasks with in-context learning. We evaluate four model families, OPT, BLOOM, CodeGen and Codex on three semantic parsing datasets.
  arXiv  Detail & Related papers  (2022-11-15T19:56:37Z)
• Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little [74.49773960145681]
  A possible explanation for the impressive performance of masked language model (MLM)-training is that such models have learned to represent the syntactic structures prevalent in NLP pipelines. In this paper, we propose a different explanation: pre-trains succeed on downstream tasks almost entirely due to their ability to model higher-order word co-occurrence statistics. Our results show that purely distributional information largely explains the success of pre-training, and underscore the importance of curating challenging evaluation datasets that require deeper linguistic knowledge.
  arXiv  Detail & Related papers  (2021-04-14T06:30:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.