Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning

• URL: http://arxiv.org/abs/2609.10445v1
• Date: Wed, 09 Sep 2026 16:56:25 GMT
• Title: Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning
• Abstract summary: Reasoning language models primarily reason in English regardless of the language they are prompted in.<n>This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language.<n>In this work, we advance L2 reasoning, the ability of a model to reason consistently in the language of the user's prompt, thus building an in-language bridge between the prompt and the answer.
• Score: 75.46846056993418
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: models primarily reason in English regardless of the language they are prompted in. This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language. In this work, we advance L2 reasoning, the ability of a model to reason consistently in the language of the user's prompt, thus building an in-language bridge between the prompt and the answer. We approach this problem from a data-centric angle, investigating how to optimize data composition and scheduling in SFT for reasoning generalization. Building Tiny Aya L2-Thinker at 3.35B scale, we achieve an L2 reasoning rate above 93% across 60 languages on 6 benchmarks spanning math, commonsense reasoning, instruction following, open-ended generation, and cultural reasoning while keeping performance strong. We show the path to generalizing L2 reasoning to held-out languages goes through broader language coverage, readily available multilingual non-reasoning data, and a sufficient English reasoning backbone. These findings indicate that reasoning is a language-agnostic behavior that can be transferred across typologically diverse languages through careful data mixing and without requiring reasoning supervision in every target language. We release our model weights and multilingual reasoning data to support further research on accessible, in-language reasoning.

Related papers

• What Makes Good Multilingual Reasoning? Disentangling Reasoning Traces with Measurable Features [32.31269959716733]
  This work asks what actually characterizes effective reasoning in multilingual settings.<n>We first define a suite of measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow aspects of reasoning traces.<n>We then train sparse autoencoders over multilingual traces to automatically discover latent reasoning concepts that instantiate or extend these features.
  arXiv  Detail & Related papers  (2026-04-06T14:40:50Z)
• Align to the Pivot: Dual Alignment with Self-Feedback for Multilingual Math Reasoning [71.4175109189942]
  We present Pivot-Aligned Self-Feedback Multilingual Reasoning (PASMR)<n>This approach designates the model's primary language as the pivot language.<n>It establishes a cross-lingual self-feedback mechanism without relying on external correct answers or reward models.
  arXiv  Detail & Related papers  (2026-01-25T03:20:00Z)
• Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning [39.03934159726098]
  M2A is a novel method that combines multi-scale multilingual alignment with language-consistency rewards on machine-translated questions.<n>We introduce GeoFact-X, a geography-based multilingual factual reasoning benchmark together with reasoning traces in five languages.<n>Our results show that M2A significantly enhances multilingual reasoning fidelity in both mathematical and factual reasoning tasks.
  arXiv  Detail & Related papers  (2025-07-07T19:04:36Z)
• EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoning [12.511775058257328]
  We investigate whether English is the most token-efficient language for reasoning.<n>We find that reasoning in non-English languages not only reduces token usage, but also preserves accuracy.<n>The extent of improvement depends on the models multilingual strength.
  arXiv  Detail & Related papers  (2025-06-30T20:29:52Z)
• MMATH: A Multilingual Benchmark for Mathematical Reasoning [94.05289799605957]
  We introduce MMATH, a benchmark for multilingual complex reasoning spanning 374 high-quality math problems across 10 typologically diverse languages.<n>We observe that even advanced models like DeepSeek R1 exhibit substantial performance disparities across languages and suffer from a critical off-target issue-generating responses in unintended languages.<n>Our findings offer new insights and practical strategies for advancing the multilingual reasoning capabilities of large language models.
  arXiv  Detail & Related papers  (2025-05-25T12:47:39Z)
• Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models? [59.970391602080205]
  Despite multilingual training, LRMs tend to default to reasoning in high-resource languages at test time.<n>Cultural reasoning degrades performance on reasoning tasks but benefits cultural tasks, while safety evaluations exhibit language-specific behavior.
  arXiv  Detail & Related papers  (2025-05-23T02:46:18Z)
• When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners [111.50503126693444]
  We show that language-specific ablation consistently boosts multilingual reasoning performance.<n>Compared to post-training, our training-free ablation achieves comparable or superior results with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-05-21T08:35:05Z)
• Crosslingual Reasoning through Test-Time Scaling [51.55526326294275]
  We find that scaling up inference compute for English-centric reasoning language models (RLMs) improves multilingual mathematical reasoning across many languages.<n>While English-centric RLM's CoTs are naturally predominantly English, they consistently follow a quote-and-think pattern to reason about quoted non-English inputs.<n>We observe poor out-of-domain reasoning generalization, in particular from STEM to cultural commonsense knowledge, even for English.
  arXiv  Detail & Related papers  (2025-05-08T16:50:06Z)
• Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention [18.439771003766026]
  We study whether multilingual language models (MultiLMs) can transfer logical reasoning abilities to other languages when they are fine-tuned for reasoning in a different language. We demonstrate that although MultiLMs can transfer reasoning ability across languages in a monolingual setting, they struggle to transfer reasoning abilities in a code-switched setting. Following this observation, we propose a novel attention mechanism that uses a dedicated set of parameters to encourage cross-lingual attention in code-switched sequences.
  arXiv  Detail & Related papers  (2023-10-23T18:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.