Efficient Multilingual Reasoning Transfer via Progressive Code-Switching

• URL: http://arxiv.org/abs/2607.00485v1
• Date: Wed, 01 Jul 2026 06:13:16 GMT
• Title: Efficient Multilingual Reasoning Transfer via Progressive Code-Switching
• Abstract summary: Transfer approaches typically rely on distilled target-language reasoning traces from stronger LRMs or online supervision from external judge models.<n>We propose PCS (Progressive Code-Switching), a more efficient transfer framework that requires only lightweight translation.<n> PCS substantially narrows the performance gap between target-language and English reasoning, yielding more language-consistent reasoning.
• Score: 51.49388869532955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large reasoning models (LRMs) have achieved strong reasoning capabilities in English, yet their performance degrades significantly when required to reason in other languages. A natural solution is to transfer the model's English reasoning ability to target languages. However, existing transfer approaches typically rely on distilled target-language reasoning traces from stronger LRMs or online supervision from external judge models, which are costly and difficult to scale. In this paper, we propose PCS (Progressive Code-Switching), a more efficient transfer framework that requires only lightweight translation without any stronger model for distillation or judging. PCS first constructs code-switched reasoning traces by translating a subset of English reasoning steps into the target language, and uses them to initialize the model's code-switching ability via supervised fine-tuning. It then applies reinforcement learning with a step-level language consistency curriculum, progressively raising the target-language ratio until the model reasons entirely in the target language. This progressive design provides a smooth transfer path that avoids the instability and performance degradation commonly observed when directly enforcing target-language reasoning. Experiments on multiple benchmarks and five typologically diverse languages show that PCS substantially narrows the performance gap between target-language and English reasoning, yielding more language-consistent reasoning while maintaining competitive accuracy.

Related papers

• AdaMame: A Training Recipe for Adaptive Multilingual Reasoning [32.31269959716733]
  Large Reasoning Models (LRMs) often fail to reason in the language of the query.<n>AdaMame is a two-stage training recipe for multilingual mathematical reasoning.
  arXiv  Detail & Related papers  (2026-06-13T03:22:35Z)
• Learning When to Translate for Multilingual Reasoning [20.51941069559769]
  Reasoning language models (RLMs) achieve strong performance on complex reasoning tasks, but still exhibit substantial multilingual reasoning gaps.<n>English translation can mitigate these failures by expressing non-English inputs in a form that RLMs can more reliably interpret.<n>We propose Luar, a Language Understanding Boundary-aware Reinforcement Learning framework that trains RLMs to selectively invoke translation when direct understanding is unreliable.
  arXiv  Detail & Related papers  (2026-06-01T16:37:18Z)
• LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance [77.58408743830314]
  Reinforcement learning has proven effective for enhancing multi-step reasoning in large language models.<n>But its benefits have not fully translated to multilingual contexts.<n>We develop a novel framework that leverages language-conditioned hints to guide exploration in non-English reasoning tasks.
  arXiv  Detail & Related papers  (2026-05-21T14:47:52Z)
• ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance [44.67427662307542]
  We introduce ReasonXL, the first large-scale parallel corpus of cross-domain reasoning traces spanning five European languages.<n>Using ReasonXL, we demonstrate that LLMs can be adapted to reason entirely in a desired target language.<n>We conduct an extensive representational analysis of the adaptation and find a clear functional division across model depth.
  arXiv  Detail & Related papers  (2026-04-14T07:10:46Z)
• Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning [85.7304930030649]
  We propose M-Thinker, which is trained by a Language Consistency reward and a Cross-lingual Thinking Alignment reward.<n>M-Thinker achieves nearly 100% language consistency and superior performance on two multilingual benchmarks.
  arXiv  Detail & Related papers  (2025-10-08T17:55:02Z)
• Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models [64.54005959758733]
  We introduce code-switching in-context learning (CSICL) as a principled and robust approach for overcoming the translation barrier during inference.<n>We conduct extensive experiments across 4 LLMs, 6 datasets, and 10 languages, spanning both knowledge-intensive and reasoning-oriented domains.<n>Our results demonstrate CSICL consistently outperforms X-ICL baselines, achieving gains of 3.1%p and 1.9%p in both target and unseen languages.
  arXiv  Detail & Related papers  (2025-10-07T08:35:42Z)
• Long Chain-of-Thought Reasoning Across Languages [14.79632337642471]
  We investigate four key stages of model development: scaling, pretraining, post-training, and inference.<n>We find that scaling reasoning model size improves multilingual task performance in En-CoT, but Target-CoT performance lags behind.<n>Given the scarcity of high-quality reasoning traces in languages other than English, we explore synthetic data curation approaches for post-training.
  arXiv  Detail & Related papers  (2025-08-20T16:22:51Z)
• When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners [111.50503126693444]
  We show that language-specific ablation consistently boosts multilingual reasoning performance.<n>Compared to post-training, our training-free ablation achieves comparable or superior results with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-05-21T08:35:05Z)
• AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thought [40.16140566668239]
  We introduce AdaMCOT, a framework that enhances multilingual factual reasoning.<n>AdaMCOT dynamically routing thought processes in intermediary "thinking languages" before generating target-language responses.<n>Our evaluation demonstrates substantial improvements in both factual reasoning quality and cross-lingual consistency.
  arXiv  Detail & Related papers  (2025-01-27T15:48:57Z)
• From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers [62.637055980148816]
  Massively multilingual transformers pretrained with language modeling objectives have become a de facto default transfer paradigm for NLP. We show that cross-lingual transfer via massively multilingual transformers is substantially less effective in resource-lean scenarios and for distant languages.
  arXiv  Detail & Related papers  (2020-05-01T22:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.