Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Models

• URL: http://arxiv.org/abs/2604.06202v1
• Date: Fri, 13 Mar 2026 21:47:50 GMT
• Title: Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Models
• Abstract summary: Large language models (LLMs) have transformed natural language processing, yet their capabilities remain uneven across languages.<n>Most multilingual models are trained primarily on high-resource languages, leaving many languages with large speaker populations underrepresented in both training data and evaluation benchmarks.<n>This paper proposes a theoretical framework for studying cross-lingual transfer and parameter-efficient adaptation of multilingual LLMs within the Turkic language family.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have transformed natural language processing, yet their capabilities remain uneven across languages. Most multilingual models are trained primarily on high-resource languages, leaving many languages with large speaker populations underrepresented in both training data and evaluation benchmarks. This imbalance is particularly visible in the Turkic language family. This paper proposes a theoretical framework for studying cross-lingual transfer and parameter-efficient adaptation of multilingual LLMs within the Turkic language family, focusing on Azerbaijani, Kazakh, Uzbek, Turkmen, and Gagauz. These languages share substantial typological and morphological similarity while differing greatly in available digital resources, making them a natural setting for analyzing multilingual adaptation strategies. We integrate insights from multilingual representation learning and parameter-efficient fine-tuning techniques such as Low-Rank Adaptation (LoRA) to develop a conceptual scaling model describing how adaptation performance depends on model capacity, adaptation data size, and the expressivity of adaptation modules. To formalize transfer potential between related languages, we introduce the Turkic Transfer Coefficient (TTC), a theoretical measure incorporating morphological similarity, lexical overlap, syntactic structure, and script compatibility across Turkic languages. The framework highlights how typological similarity can enable efficient multilingual transfer while also identifying structural limits of parameter-efficient adaptation in extremely low-resource scenarios.

Related papers

• MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition [23.519328539637794]
  MoLGE assigns dedicated expert modules to clusters of similar languages.<n>It integrates a hierarchical Low-Rank Adaptation (LoRA) strategy into the disentangled acoustic and linguistic components of the S3M architecture.<n>Results demonstrate that MoLGE consistently outperforms dense multilingual baselines with a minimal increase in trainable parameters.
  arXiv  Detail & Related papers  (2026-07-27T05:55:50Z)
• Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weights [5.014897572890602]
  Large Language Models (LLMs) remain heavily centered on English, with limited performance in low-resource languages.<n>We show that model merging is a viable and efficient alternative to traditional adaptation methods for low-resource languages.
  arXiv  Detail & Related papers  (2026-03-30T10:46:50Z)
• Typologically-Informed Candidate Reranking for LLM-based Translation into Low-Resource Languages [1.0732935873226022]
  Large language models trained predominantly on high-resource languages exhibit systematic biases toward dominant typological patterns.<n>We present a framework that leverages linguistic typology to improve translation quality without parallel training data or model retraining.
  arXiv  Detail & Related papers  (2026-02-01T11:22:30Z)
• Analyzing and Improving Cross-lingual Knowledge Transfer for Machine Translation [5.878901309908815]
  We study cross-lingual knowledge transfer in neural models and develop methods to improve robustness and generalization in multilingual settings.<n>We examine the role of language diversity during training and show that increasing translation coverage improves generalization and reduces off-target behavior.
  arXiv  Detail & Related papers  (2026-01-07T15:51:54Z)
• Cross-Linguistic Transfer in Multilingual NLP: The Role of Language Families and Morphology [0.0]
  Cross-lingual transfer has become a crucial aspect of multilingual NLP.<n>This paper investigates cross-linguistic transfer through the lens of language families and morphology.
  arXiv  Detail & Related papers  (2025-05-20T04:19:34Z)
• MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting [53.77590764277568]
  We introduce a novel MoE-CT architecture that separates the base model's learning from the multilingual expansion process. Our design freezes the original LLM parameters, thus safeguarding its performance in high-resource languages, while an appended MoE module, trained on diverse language datasets, augments low-resource language proficiency.
  arXiv  Detail & Related papers  (2024-06-25T11:03:45Z)
• Targeted Multilingual Adaptation for Low-resource Language Families [17.212424929235624]
  We study best practices for adapting a pre-trained model to a language family. Our adapted models significantly outperform mono- and multilingual baselines. Low-resource languages can be aggressively up-sampled during training at little detriment to performance in high-resource languages.
  arXiv  Detail & Related papers  (2024-05-20T23:38:06Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning [68.57658225995966]
  Cross-lingual Choice of Plausible Alternatives (XCOPA) is a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods falls short compared to translation-based transfer.
  arXiv  Detail & Related papers  (2020-05-01T12:22:33Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)
• Cross-lingual, Character-Level Neural Morphological Tagging [57.0020906265213]
  We train character-level recurrent neural taggers to predict morphological taggings for high-resource languages and low-resource languages together.<n>Learning joint character representations among multiple related languages successfully enables knowledge transfer from the high-resource languages to the low-resource ones, improving accuracy by up to 30% over a monolingual model.
  arXiv  Detail & Related papers  (2017-08-30T08:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.