More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs

• URL: http://arxiv.org/abs/2608.30463v1
• Date: Mon, 31 Aug 2026 08:49:51 GMT
• Title: More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs
• Abstract summary: We introduce the first multilingual benchmark of paired solvable and unsolvable mathematical problems.<n>We analyze the solvability detection capabilities of state-of-the-art Large Language Models.<n>We find that Solvability Belief is encoded as a largely universal, language-agnostic feature.
• Score: 5.831893840514845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solvability detection is one of the most challenging aspects of mathematical reasoning for Large Language Models (LLMs). While prior work has studied this capability extensively, these analyses have been limited to English. Consequently, it remains unclear whether multilingual failures arise from differences in internal Solvability Belief or from language-dependent failures to express it. To address this gap, we introduce the first multilingual benchmark of paired solvable and unsolvable mathematical problems, extending ReliableMath to French and Greek. Using this, we train multilingual probes predicting Solvability Belief and analyze the solvability detection capabilities of state-of-the-art LLMs behaviorally, representationally, and in terms of faithfulness. We find that Solvability Belief is encoded as a largely universal, language-agnostic feature, and that higher-resource languages such as English, despite achieving stronger mathematical reasoning performance, exhibit lower solvability-detection faithfulness.

Related papers

• LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance [77.58408743830314]
  Reinforcement learning has proven effective for enhancing multi-step reasoning in large language models.<n>But its benefits have not fully translated to multilingual contexts.<n>We develop a novel framework that leverages language-conditioned hints to guide exploration in non-English reasoning tasks.
  arXiv  Detail & Related papers  (2026-05-21T14:47:52Z)
• Beyond Translation: Evaluating Mathematical Reasoning Capabilities of LLMs in Sinhala and Tamil [1.0499611180329804]
  Large language models (LLMs) demonstrate strong mathematical reasoning in English.<n>But whether these capabilities reflect genuine multilingual reasoning or reliance on translation-based processing in low-resource languages like Sinhala and Tamil remains unclear.<n>We evaluate four prominent large language models using a taxonomy of six math problem types.
  arXiv  Detail & Related papers  (2026-02-16T07:08:37Z)
• Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners [48.68444770923683]
  Large reasoning models (LRMs) achieve strong performance on mathematical reasoning tasks.<n>LRMs often arrive at the correct answer before completing these textual reasoning steps.<n>This phenomenon has been explored in English, but its multilingual behavior remains largely unknown.
  arXiv  Detail & Related papers  (2026-01-06T13:20:17Z)
• Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models? [18.99223776816893]
  Reasoning language models (RLMs) achieve strong performance on complex reasoning tasks, yet they still suffer from a multilingual reasoning gap.<n>This paper shows that the multilingual reasoning gap largely stems from failures in language understanding.<n>We propose Selective Translation, a simple yet effective strategy that translates the multilingual input into English only when an understanding failure is detected.
  arXiv  Detail & Related papers  (2025-10-31T08:17:59Z)
• MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning [6.8892368960722346]
  We introduce MathMist, a parallel multilingual benchmark for mathematical problem solving and reasoning.<n>MathMist encompasses over 21K aligned question-answer pairs across seven languages.<n>We systematically evaluate a diverse suite of models, including open-source small and medium LLMs, proprietary systems, and multilingual-reasoning-focused models.
  arXiv  Detail & Related papers  (2025-10-16T04:59:52Z)
• A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages [48.68444770923683]
  We present the first comprehensive study of multilingual Chain-of-Thought (CoT) reasoning.<n>We measure language compliance, answer accuracy, and answer consistency when LRMs are prompt-hacked to think in a target language.<n>We find that the quality and effectiveness of thinking traces vary substantially depending on the prompt language.
  arXiv  Detail & Related papers  (2025-10-10T17:06:50Z)
• From Logic to Language: A Trust Index for Problem Solving with LLMs [0.0]
  This paper introduces a unified framework to understand and contrast Large Language Models (LLMs)<n>We define and delineate the problem spaces addressable by formal languages versus natural language.<n>We therefore introduce a vector-valued trust index Q, which reflects solution quality and distinguishes the binary correctness of formal solutions from the continuous adequacy spectrum characteristic of natural language solutions.
  arXiv  Detail & Related papers  (2025-07-21T19:50:45Z)
• Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing [66.04926909181653]
  We argue for anthropomimetic uncertainty, meaning that intuitive and trustworthy uncertainty communication requires a degree of linguistic authenticity and personalization to the user.<n>We conclude by pointing out unique factors in human-machine communication of uncertainty and deconstruct the data biases that influence machine uncertainty communication.
  arXiv  Detail & Related papers  (2025-07-11T14:07:22Z)
• Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models [55.14276067678253]
  This paper introduces a novel methodology for efficiently identifying inherent cross-lingual weaknesses in Large Language Models (LLMs)<n>We construct a new dataset of over 6,000 bilingual pairs across 16 languages using this methodology, demonstrating its effectiveness in revealing weaknesses even in state-of-the-art models.<n>Further experiments investigate the relationship between linguistic similarity and cross-lingual weaknesses, revealing that linguistically related languages share similar performance patterns.
  arXiv  Detail & Related papers  (2025-05-24T12:31:27Z)
• When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners [111.50503126693444]
  We show that language-specific ablation consistently boosts multilingual reasoning performance.<n>Compared to post-training, our training-free ablation achieves comparable or superior results with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-05-21T08:35:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.