Is Cross-Lingual Transfer in Bilingual Models Human-Like? A Study with Overlapping Word Forms in Dutch and English

• URL: http://arxiv.org/abs/2604.07067v1
• Date: Wed, 08 Apr 2026 13:17:59 GMT
• Title: Is Cross-Lingual Transfer in Bilingual Models Human-Like? A Study with Overlapping Word Forms in Dutch and English
• Abstract summary: Cognates (friends) typically lead to facilitation, whereas interlingual homographs (false friends) cause interference or no effect.<n>We examine whether cross-lingual activation in bilingual language models mirrors these patterns.
• Score: 12.996963143295654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bilingual speakers show cross-lingual activation during reading, especially for words with shared surface form. Cognates (friends) typically lead to facilitation, whereas interlingual homographs (false friends) cause interference or no effect. We examine whether cross-lingual activation in bilingual language models mirrors these patterns. We train Dutch-English causal Transformers under four vocabulary-sharing conditions that manipulate whether (false) friends receive shared or language-specific embeddings. Using psycholinguistic stimuli from bilingual reading studies, we evaluate the models through surprisal and embedding similarity analyses. The models largely maintain language separation, and cross-lingual effects arise primarily when embeddings are shared. In these cases, both friends and false friends show facilitation relative to controls. Regression analyses reveal that these effects are mainly driven by frequency rather than consistency in form-meaning mapping. Only when just friends share embeddings are the qualitative patterns of bilinguals reproduced. Overall, bilingual language models capture some cross-linguistic activation effects. However, their alignment with human processing seems to critically depend on how lexical overlap is encoded, possibly limiting their explanatory adequacy as models of bilingual reading.

Related papers

• Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models [2.475536483276974]
  We use language model training as a method for simulating a variety of highly controlled exposure conditions.<n>We create matched 100M-word mono- and bilingual datasets using synthetic data and machine translation.<n>Across model scales and measures, bilingual models perform similarly to monolingual models in one language, but show strong performance in the second language as well.
  arXiv  Detail & Related papers  (2026-03-31T10:32:25Z)
• When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training [57.230355403478995]
  We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM.<n>We find that shared concept spaces emerge early and continue to refine, but that alignment with them is language-dependent.<n>In contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior.
  arXiv  Detail & Related papers  (2026-01-30T11:23:01Z)
• False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models [53.01170039144264]
  Subword tokenizers trained on multilingual corpora naturally produce overlapping tokens across languages.<n>Does token overlap facilitate cross-lingual transfer or instead introduce interference between languages?<n>We find that models with overlap outperform models with disjoint vocabularies.
  arXiv  Detail & Related papers  (2025-09-23T07:47:54Z)
• On the Acquisition of Shared Grammatical Representations in Bilingual Language Models [6.266732217239363]
  Crosslingual transfer is crucial to contemporary language models' multilingual capabilities, but how it occurs is not well understood.<n>We ask what happens to a monolingual language model when it begins to be trained on a second language.<n>To find evidence of shared multilingual representations, we turn to structural priming, a method used to study grammatical representations in humans.
  arXiv  Detail & Related papers  (2025-03-05T23:27:58Z)
• mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models? [15.90185747024602]
  We propose a synthetic task, Multilingual Othello (mOthello), as a testbed to delve into two questions. We find that models trained with naive multilingual pretraining fail to learn a language-neutral representation across all input languages. We propose a novel approach - multilingual pretraining with unified output space - that both induces the learning of language-neutral representation and facilitates cross-lingual transfer.
  arXiv  Detail & Related papers  (2024-04-18T18:03:08Z)
• Crosslingual Structural Priming and the Pre-Training Dynamics of Bilingual Language Models [6.845954748361076]
  We use structural priming to test for abstract grammatical representations with causal effects on model outputs. We extend the approach to a Dutch-English bilingual setting, and we evaluate a Dutch-English language model during pre-training. We find that crosslingual structural priming effects emerge early after exposure to the second language, with less than 1M tokens of data in that language.
  arXiv  Detail & Related papers  (2023-10-11T22:57:03Z)
• Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure [54.01613740115601]
  We study three language properties: constituent order, composition and word co-occurrence. Our main conclusion is that the contribution of constituent order and word co-occurrence is limited, while the composition is more crucial to the success of cross-linguistic transfer.
  arXiv  Detail & Related papers  (2022-03-16T07:09:35Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment [59.995385574274785]
  We show that, contrary to previous belief, negative interference also impacts low-resource languages. We present a meta-learning algorithm that obtains better cross-lingual transferability and alleviates negative interference.
  arXiv  Detail & Related papers  (2020-10-06T20:48:58Z)
• On the Language Neutrality of Pre-trained Multilingual Representations [70.93503607755055]
  We investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings. We show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences.
  arXiv  Detail & Related papers  (2020-04-09T19:50:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.