Cost of Reasoning in non-English Languages: A Case Study on Japanese

• URL: http://arxiv.org/abs/2607.10114v1
• Date: Sat, 11 Jul 2026 04:30:43 GMT
• Title: Cost of Reasoning in non-English Languages: A Case Study on Japanese
• Abstract summary: Reasoning Language Models (RLMs) achieve their strongest performance when they reason in English.<n>We develop a Japanese-reasoning variant of Qwen-3-Swallow-8B, which is a Japanese LLM continually pretrained from Qwen-3-8B.<n>We evaluate the trained model on Japanese cultural benchmarks and observe that the model's performance is worse than the baseline models.
• Score: 16.295305195753723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning Language Models (RLMs) achieve their strongest performance when they reason in English, the language for which reasoning-oriented training data is most abundant. However, reasoning trace is a clue for model interpretability and safety, and useful in practice for both the model users and for model developers. Thus, it is desirable to be able to develop a model that reasons in a language of the user's choice, while still maintaining strong reasoning performance. To this end, we study the feasibility of training a model that reasons in Japanese. We develop a Japanese-reasoning variant of Qwen-3-Swallow-8B, which is a Japanese LLM continually pretrained from Qwen-3-8B, with GRPO and evaluate it across coding, math, and science benchmarks. The study shows that reasoning-language control is feasible by training a Japanese continually pretrained model with GRPO. However, its performance is at best on par with strong English-reasoning baselines on several benchmarks. We also evaluate the trained model on Japanese cultural benchmarks and observe that the model's performance is worse than the baseline models, suggesting that the reasoning in Japanese does not immediately improve performance on culturally relevant tasks for free.

Related papers

• Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning [75.46846056993418]
  Reasoning language models primarily reason in English regardless of the language they are prompted in.<n>This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language.<n>In this work, we advance L2 reasoning, the ability of a model to reason consistently in the language of the user's prompt, thus building an in-language bridge between the prompt and the answer.
  arXiv  Detail & Related papers  (2026-09-09T16:56:25Z)
• Align to the Pivot: Dual Alignment with Self-Feedback for Multilingual Math Reasoning [71.4175109189942]
  We present Pivot-Aligned Self-Feedback Multilingual Reasoning (PASMR)<n>This approach designates the model's primary language as the pivot language.<n>It establishes a cross-lingual self-feedback mechanism without relying on external correct answers or reward models.
  arXiv  Detail & Related papers  (2026-01-25T03:20:00Z)
• Proceedings of the Second International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2025) [15.356983665372718]
  Reasoning is an essential component of human intelligence in that it plays a fundamental role in our ability to think critically.<n>Recent leap forward in natural language processing, with the emergence of language models based on transformers, is hinting at the possibility that these models exhibit reasoning abilities.<n>It is still not easy to articulate to what extent these models are actually capable of reasoning.
  arXiv  Detail & Related papers  (2025-11-12T03:19:53Z)
• Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought [23.847410628315544]
  We introduce **Language-Mixed CoT**, a reasoning schema that switches between English and a target language.<n>We train ninve models (4B-35B) across six families (Qwen2.5, Llama-3.1, Gemma-3, etc)<n>Our best model, **KO-REAson-35B**, achieves state-of-the-art performance, with the highest overall average score (64.0 pm 25)
  arXiv  Detail & Related papers  (2025-10-05T14:39:41Z)
• mR3: Multilingual Rubric-Agnostic Reward Reasoning Models [16.953894896444403]
  We introduce mR3, a massively multilingual, rubric-agnostic reward reasoning model trained on 72 languages.<n>We present a comprehensive study of data and curriculum selection for training to identify effective strategies and data sources for building high-quality reward models.<n>Our approach attains state-of-the-art performance on multilingual reward model benchmarks, surpassing much larger models.
  arXiv  Detail & Related papers  (2025-10-01T17:36:59Z)
• Proceedings of the First International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2024) [16.282850445579857]
  Reasoning is an essential component of human intelligence as it plays a fundamental role in our ability to think critically. Recent leap forward in natural language processing, with the emergence of language models based on transformers, is hinting at the possibility that these models exhibit reasoning abilities. Despite ongoing discussions about what reasoning is in language models, it is still not easy to pin down to what extent these models are actually capable of reasoning.
  arXiv  Detail & Related papers  (2024-10-07T02:31:47Z)
• The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights [108.40766216456413]
  We propose a question alignment framework to bridge the gap between large language models' English and non-English performance. Experiment results show it can boost multilingual performance across diverse reasoning scenarios, model families, and sizes. We analyze representation space, generated response and data scales, and reveal how question translation training strengthens language alignment within LLMs.
  arXiv  Detail & Related papers  (2024-05-02T14:49:50Z)
• ALERT: Adapting Language Models to Reasoning Tasks [43.8679673685468]
  ALERT is a benchmark and suite of analyses for assessing language models' reasoning ability. ALERT provides a test bed to asses any language model on fine-grained reasoning skills. We find that language models learn more reasoning skills during finetuning stage compared to pretraining state.
  arXiv  Detail & Related papers  (2022-12-16T05:15:41Z)
• Reasoning Like Program Executors [48.819113224699976]
  POET empowers language models to harvest the reasoning knowledge possessed in program executors via a data-driven approach. PoET can significantly boost model performance on natural language reasoning. PoET opens a new gate on reasoning-enhancement pre-training.
  arXiv  Detail & Related papers  (2022-01-27T12:28:24Z)
• From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension [51.953428342923885]
  We develop a two-stage approach to enhance the model performance. The first stage targets at recall: we design a hard-learning (HL) algorithm to maximize the likelihood that the top-k predictions contain the accurate answer. The second stage focuses on precision: an answer-aware contrastive learning mechanism is developed to learn the fine difference between the accurate answer and other candidates.
  arXiv  Detail & Related papers  (2021-12-09T07:31:15Z)
• Pre-Training a Language Model Without Human Language [74.11825654535895]
  We study how the intrinsic nature of pre-training data contributes to the fine-tuned downstream performance. We find that models pre-trained on unstructured data beat those trained directly from scratch on downstream tasks. To our great astonishment, we uncover that pre-training on certain non-human language data gives GLUE performance close to performance pre-trained on another non-English language.
  arXiv  Detail & Related papers  (2020-12-22T13:38:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.