Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

• URL: http://arxiv.org/abs/2605.19723v1
• Date: Tue, 19 May 2026 11:56:03 GMT
• Title: Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
• Abstract summary: This survey synthesizes recent advancements in mathematical reasoning with Large Language Models (LLMs)<n>Our systematic review encompasses approximately 120 peer-reviewed studies and preprints, examining the evolution of this research area.
• Score: 1.8499314936771558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems. As Large Language Models (LLMs) improve their reasoning capabilities, understanding how well they perform mathematical reasoning has become increasingly important. This survey synthesizes recent advancements in mathematical reasoning with LLMs through a structured analysis of datasets, architectures, training strategies, and evaluation protocols. Our systematic review encompasses approximately 120 peer-reviewed studies and preprints, examining the evolution of this research area and providing a unified analytical framework to understand current progress and limitations. Our study particularly introduces a unified taxonomy of mathematical datasets, distinguishing between pretraining corpora, supervised fine-tuning resources, and evaluation benchmarks across varying levels of reasoning complexity. A systematic analysis of reasoning architectures and training strategies, including tool integration, verifier-guided reasoning, and parameter-efficient adaptation, is presented to assess their effects on reasoning robustness and generalization. Moreover, a comparative evaluation of existing metrics highlights the gap between final-answer accuracy and process-level reasoning verification. By synthesizing insights across these areas, our analysis identifies recurring failure modes, such as reasoning faithfulness issues, benchmark biases, and generalization limitations, and outlines key research directions toward improving symbolic grounding, evaluation reliability, and the development of more robust and trustworthy LLM-based reasoning systems.

Related papers

• Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms [16.041484242548837]
  We introduce the theoretical foundations of scaling laws in reinforcement learning.<n>We show that the performance of reinforcement learning algorithms does not have a monotone relationship between performance rankings and data-regimes.
  arXiv  Detail & Related papers  (2026-07-08T16:12:14Z)
• LLM-Based Scientific Peer Review: Methods, Benchmarks, and Reliability Challenges [4.655159257282136]
  This survey offers a systems-level analysis of large language models (LLMs)-based scientific peer review.<n>We focus on two core evaluative functions: critique generation and score prediction.<n>We identify emerging risks, including prompt injection, data poisoning, retrieval vulnerabilities, and reward hacking.
  arXiv  Detail & Related papers  (2026-06-23T18:12:39Z)
• The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes [55.42597391158023]
  This survey provides a systematic analysis of more than 300 recent papers from arXiv, Semantic Scholar, Google Scholar, Papers with Code, and the ACL Anthology.<n>We introduce a structured taxonomy of reasoning research, covering Chain-of-Thought reasoning, multi-hop reasoning, mathematical reasoning, common sense reasoning, visual and temporal reasoning, code and algorithmic reasoning, retrieval-augmented reasoning, tool-augmented and agentic reasoning, and reinforcement learning-based reasoning.<n>We synthesize recurring limitations and failure modes, such as reasoning hallucinations, brittle multi-step inference, weak causal abstraction, and poor cross
  arXiv  Detail & Related papers  (2026-06-09T21:59:37Z)
• Prediction Model of Motivators and Demotivators of Integrating Large Language Models in Software Engineering Education: An Empirical Study [0.9549646359252346]
  Large Language Models (LLMs) are increasingly influencing software engineering practice and education.<n>This study develops and validates a prediction model to identify cost-efficient strategies for integrating LLMs into software engineering education.
  arXiv  Detail & Related papers  (2026-05-10T07:41:25Z)
• Schoenfeld's Anatomy of Mathematical Reasoning by Language Models [56.656180566692946]
  We adopt Schoenfeld's Episode Theory as an inductive, intermediate-scale lens and introduce ThinkARM (Anatomy of Reasoning in Models)<n>ThinkARM explicitly abstracts reasoning traces into functional reasoning steps such as Analysis, Explore, Implement, verify, etc.<n>We show that episode-level representations make reasoning steps explicit, enabling systematic analysis of how reasoning is structured, stabilized, and altered in modern language models.
  arXiv  Detail & Related papers  (2025-12-23T02:44:25Z)
• Interpretability Framework for LLMs in Undergraduate Calculus [0.0]
  Large Language Models (LLMs) are increasingly being used in education, yet their correctness alone does not capture the quality, reliability, or pedagogical validity of their problem-solving behavior.<n>We introduce a novel interpretability framework for analyzing LLM-generated solutions using undergraduate calculus problems as a representative domain.<n>Our approach combines reasoning flow extraction and decomposing solutions into semantically labeled operations and concepts with prompt ablation analysis to assess input salience and output stability.
  arXiv  Detail & Related papers  (2025-10-19T17:20:36Z)
• PRISM-Physics: Causal DAG-Based Process Evaluation for Physics Reasoning [57.868248683256574]
  PRISM-Physics is a process-level evaluation framework and benchmark for complex physics reasoning problems.<n> Solutions are represented as directed acyclic graphs (DAGs) of formulas.<n>Results show that our evaluation framework is aligned with human experts' scoring.
  arXiv  Detail & Related papers  (2025-10-03T17:09:03Z)
• Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization [1.246870021158888]
  This paper investigates the capabilities of large language models (LLMs) in formulating and solving decision-making problems using mathematical programming.<n>We first conduct a systematic review and meta-analysis of recent literature to assess how well LLMs understand, structure, and solve optimization problems across domains.<n>Our systematic evidence is complemented by targeted experiments designed to evaluate the performance of state-of-the-art LLMs in automatically generating optimization models for problems in computer networks.
  arXiv  Detail & Related papers  (2025-08-25T14:52:56Z)
• Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications [59.721265428780946]
  Large Language Models (LLMs) in medicine have enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning.<n>This paper provides the first systematic review of this emerging field.<n>We propose a taxonomy of reasoning enhancement techniques, categorized into training-time strategies and test-time mechanisms.
  arXiv  Detail & Related papers  (2025-08-01T14:41:31Z)
• On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study [15.617243755155686]
  Counterfactual reasoning has emerged as a crucial technique for generalizing the reasoning capabilities of large language models.<n>We propose a decompositional strategy that breaks down the counterfactual generation from causality construction to the reasoning over counterfactual interventions.
  arXiv  Detail & Related papers  (2025-05-17T04:59:32Z)
• Evaluating Mathematical Reasoning Beyond Accuracy [50.09931172314218]
  We introduce ReasonEval, a new methodology for evaluating the quality of reasoning steps.<n>We show that ReasonEval consistently outperforms baseline methods in the meta-evaluation datasets.<n>We observe that ReasonEval can play a significant role in data selection.
  arXiv  Detail & Related papers  (2024-04-08T17:18:04Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.