Analyzing and Improving Cross-lingual Knowledge Transfer for Machine Translation

• URL: http://arxiv.org/abs/2601.04036v1
• Date: Wed, 07 Jan 2026 15:51:54 GMT
• Title: Analyzing and Improving Cross-lingual Knowledge Transfer for Machine Translation
• Authors: David Stap,
• Abstract summary: We study cross-lingual knowledge transfer in neural models and develop methods to improve robustness and generalization in multilingual settings.<n>We examine the role of language diversity during training and show that increasing translation coverage improves generalization and reduces off-target behavior.
• Score: 5.878901309908815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multilingual machine translation systems aim to make knowledge accessible across languages, yet learning effective cross-lingual representations remains challenging. These challenges are especially pronounced for low-resource languages, where limited parallel data constrains generalization and transfer. Understanding how multilingual models share knowledge across languages requires examining the interaction between representations, data availability, and training strategies. In this thesis, we study cross-lingual knowledge transfer in neural models and develop methods to improve robustness and generalization in multilingual settings, using machine translation as a central testbed. We analyze how similarity between languages influences transfer, how retrieval and auxiliary supervision can strengthen low-resource translation, and how fine-tuning on parallel data can introduce unintended trade-offs in large language models. We further examine the role of language diversity during training and show that increasing translation coverage improves generalization and reduces off-target behavior. Together, this work highlights how modeling choices and data composition shape multilingual learning and offers insights toward more inclusive and resilient multilingual NLP systems.

Related papers

• Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics [56.145578792496714]
  Large language models (LLMs) struggle with cross-lingual knowledge transfer.<n>We study the causes and dynamics of this phenomenon by training small Transformer models from scratch on synthetic multilingual datasets.
  arXiv  Detail & Related papers  (2025-08-14T18:44:13Z)
• Exploring Cross-lingual Latent Transplantation: Mutual Opportunities and Open Challenges [48.96952594416528]
  Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability.<n>XTransplant framework enables models to harness the complementary strengths of both English and non-English resources by transplanting latent activations across languages.
  arXiv  Detail & Related papers  (2024-12-17T09:05:30Z)
• Towards a Deep Understanding of Multilingual End-to-End Speech Translation [52.26739715012842]
  We analyze representations learnt in a multilingual end-to-end speech translation model trained over 22 languages. We derive three major findings from our analysis.
  arXiv  Detail & Related papers  (2023-10-31T13:50:55Z)
• Language Agnostic Multilingual Information Retrieval with Contrastive Learning [59.26316111760971]
  We present an effective method to train multilingual information retrieval systems. We leverage parallel and non-parallel corpora to improve the pretrained multilingual language models. Our model can work well even with a small number of parallel sentences.
  arXiv  Detail & Related papers  (2022-10-12T23:53:50Z)
• On the cross-lingual transferability of multilingual prototypical models across NLU tasks [2.44288434255221]
  Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven to be effective for limited domain and language applications. In practice, these approaches suffer from the drawbacks of domain-driven design and under-resourced languages. This article proposes to investigate the cross-lingual transferability of using synergistically few-shot learning with prototypical neural networks and multilingual Transformers-based models.
  arXiv  Detail & Related papers  (2022-07-19T09:55:04Z)
• Cross-lingual Lifelong Learning [53.06904052325966]
  We present a principled Cross-lingual Continual Learning (CCL) evaluation paradigm. We provide insights into what makes multilingual sequential learning particularly challenging. The implications of this analysis include a recipe for how to measure and balance different cross-lingual continual learning desiderata.
  arXiv  Detail & Related papers  (2022-05-23T09:25:43Z)
• Adaptive Sparse Transformer for Multilingual Translation [18.017674093519332]
  A known challenge of multilingual models is the negative language interference. We propose an adaptive and sparse architecture for multilingual modeling. Our model outperforms strong baselines in terms of translation quality without increasing the inference cost.
  arXiv  Detail & Related papers  (2021-04-15T10:31:07Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)
• A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT [60.9051207862378]
  multilingual BERT works remarkably well on cross-lingual transfer tasks. Datasize and context window size are crucial factors to the transferability. There is a computationally cheap but effective approach to improve the cross-lingual ability of multilingual BERT.
  arXiv  Detail & Related papers  (2020-04-20T11:13:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.