There is No Theoretical Curse of Multilinguality For Embedding Space Structure

• URL: http://arxiv.org/abs/2608.17088v1
• Date: Mon, 17 Aug 2026 19:50:44 GMT
• Title: There is No Theoretical Curse of Multilinguality For Embedding Space Structure
• Abstract summary: A central goal of multilingual NLP is to achieve high monolingual performance per language and cross-lingual alignment for large-scale language coverage with a multilingual model.<n>The curse of multilinguality describes the phenomenon of degradation in multilingual model performance as we increase language coverage.<n>This paper asks whether multilingual embedding spaces are inherently incapable of achieving perfect multilinguality without a prohibitive increase in required capacity.
• Score: 21.81373742083686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A central goal of multilingual NLP is to achieve high monolingual performance per language and cross-lingual alignment for large-scale language coverage with a multilingual model. The curse of multilinguality describes the phenomenon of degradation in multilingual model performance as we increase language coverage, posing a threat to the above goal. This paper asks whether multilingual embedding spaces are inherently incapable of achieving perfect multilinguality without a prohibitive increase in required capacity. We first formalize the goal of "perfect multilinguality", embodied in two multilinguality conditions. We then prove that the minimum dimensionality required for perfect multilinguality grows only logarithmically in the number of languages. That is, we show that there is no theoretical curse of multilinguality for embedding space structure. This suggests that the empirical curse of multilinguality is a result of real world data and training conditions. We back this understanding with a small-scale empirical study. Our paper provides the first theoretical and intrinsic perspective on the curse of multilinguality, with implications for the scientific understanding of this phenomenon.

Related papers

• Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs [25.841183184960475]
  This paper argues that contemporary multilingual NLP has converged on a fragile and misleading paradigm of incidental multilingualism.<n>We contend that this paradigm systematically produces unequal, brittle, and opaque behavior across languages.<n>To address this, we call for a shift toward multilingualism by design.
  arXiv  Detail & Related papers  (2026-05-02T03:39:12Z)
• English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training [7.141617235075284]
  Post-training pipelines remain predominantly English-centric, contributing to performance disparities across languages.<n>We present a systematic, controlled study of the interplay between training language coverage, model scale, and task domain.<n>We find that increasing language coverage during post-training is largely beneficial across tasks and model scales.
  arXiv  Detail & Related papers  (2026-04-14T20:26:34Z)
• Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models [2.475536483276974]
  We use language model training as a method for simulating a variety of highly controlled exposure conditions.<n>We create matched 100M-word mono- and bilingual datasets using synthetic data and machine translation.<n>Across model scales and measures, bilingual models perform similarly to monolingual models in one language, but show strong performance in the second language as well.
  arXiv  Detail & Related papers  (2026-03-31T10:32:25Z)
• High-Dimensional Interlingual Representations of Large Language Models [65.77317753001954]
  Large language models (LLMs) trained on massive multilingual datasets hint at the formation of interlingual constructs.<n>We explore 31 diverse languages varying on their resource-levels, typologies, and geographical regions.<n>We find that multilingual LLMs exhibit inconsistent cross-lingual alignments.
  arXiv  Detail & Related papers  (2025-03-14T10:39:27Z)
• On the Acquisition of Shared Grammatical Representations in Bilingual Language Models [6.266732217239363]
  Crosslingual transfer is crucial to contemporary language models' multilingual capabilities, but how it occurs is not well understood.<n>We ask what happens to a monolingual language model when it begins to be trained on a second language.<n>To find evidence of shared multilingual representations, we turn to structural priming, a method used to study grammatical representations in humans.
  arXiv  Detail & Related papers  (2025-03-05T23:27:58Z)
• Lens: Rethinking Multilingual Enhancement for Large Language Models [70.85065197789639]
  We propose Lens, a novel approach to enhance multilingual capabilities in large language models (LLMs)<n>Lens operates on two subspaces: the language-agnostic subspace, where it aligns target languages with the central language to inherit strong semantic representations, and the language-specific subspace, where it separates target and central languages to preserve linguistic specificity.<n>Lens significantly improves multilingual performance while maintaining the model's English proficiency, achieving better results with less computational cost compared to existing post-training approaches.
  arXiv  Detail & Related papers  (2024-10-06T08:51:30Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora.<n>But can these models relate corresponding concepts across languages, i.e., be crosslingual?<n>This study evaluates state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? [34.38469832305664]
  This paper focuses on human values-related concepts (i.e., value concepts) due to their significance for AI safety. We first empirically confirm the presence of value concepts within LLMs in a multilingual format. Further analysis on the cross-lingual characteristics of these concepts reveals 3 traits arising from language resource disparities.
  arXiv  Detail & Related papers  (2024-02-28T07:18:39Z)
• Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed? [40.13166574854085]
  We investigate the minimal amount of multilinguality required to elicit cross-lingual generalisation in English-centric large language models. We find that multilingual instruction tuning with as few as two to three languages is both necessary and sufficient to elicit effective cross-lingual generalisation.
  arXiv  Detail & Related papers  (2023-12-20T00:49:52Z)
• Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure [54.01613740115601]
  We study three language properties: constituent order, composition and word co-occurrence. Our main conclusion is that the contribution of constituent order and word co-occurrence is limited, while the composition is more crucial to the success of cross-linguistic transfer.
  arXiv  Detail & Related papers  (2022-03-16T07:09:35Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages with Adversarial Examples [51.048234591165155]
  We present AM2iCo, Adversarial and Multilingual Meaning in Context. It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts. Results reveal that current SotA pretrained encoders substantially lag behind human performance.
  arXiv  Detail & Related papers  (2021-04-17T20:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.