Beyond Accuracy: Evaluating Strategy Diversity in LLM Mathematical Reasoning

• URL: http://arxiv.org/abs/2605.09292v1
• Date: Sun, 10 May 2026 03:38:37 GMT
• Title: Beyond Accuracy: Evaluating Strategy Diversity in LLM Mathematical Reasoning
• Abstract summary: Strategy-level evaluation framework instantiated on AMC 10/12 and AIME problems.<n>We find a pronounced decoupling between answer accuracy and strategy diversity.<n> Gemini, DeepSeek, GPT, and Claude generate 184, 152, 151, and 110 distinct valid strategies.
• Score: 11.576914513156316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models now achieve high final-answer accuracy on mathematical reasoning benchmarks, but accuracy alone does not capture reasoning flexibility. We introduce a strategy-level evaluation framework instantiated on 80 AMC 10/12 and AIME problems with 217 AoPS-derived reference strategy families. Model outputs are annotated for strategy identity, validity, and correctness using dual-AI coding with human adjudication. Across four frontier models, we find a pronounced decoupling between answer accuracy and strategy diversity. Under a single-solution prompt, all models achieve high accuracy (95%-100%), but under a multiple-strategy prompt they recover substantially fewer strategies than the human reference set. Gemini, DeepSeek, GPT, and Claude generate 184, 152, 151, and 110 distinct valid strategies, respectively, with the largest gaps in Geometry and Number Theory. The models collectively produce 50 benchmark-novel valid strategies, indicating both incomplete coverage of human strategies and some capacity for alternative reasoning. A repeated-run robustness check on 20 problems shows diminishing gains in discovered strategies, with the strongest model recovering only 39 of 55 AoPS-reference strategies (71%) after three runs. These findings position strategy diversity as a complementary dimension for evaluating mathematical reasoning beyond answer correctness.

Related papers

• EVOQUANT: Self-Evolving Verifier-Guided Strategy Optimization for Robust Quantitative Trading [24.38446478098508]
  EVOQUANT is a self-Evolving Verifier-guided framework for strategy optimization in Quantitative trading.<n>Our method generates semantically controlled candidate edits, selects the best strategy through a multi-stage verification pipeline.<n> Experimental results show that our method significantly improves the Sharpe ratio across all tested strategies.
  arXiv  Detail & Related papers  (2026-07-14T07:34:01Z)
• StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction [76.50439715778869]
  StraTA is a framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning.<n>Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance.
  arXiv  Detail & Related papers  (2026-05-07T17:51:16Z)
• Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance [86.46794021499511]
  We show a previously underexplored gap between strategy usage and strategy executability.<n>We propose Selective Strategy Retrieval (SSR), a test-time framework that explicitly models executability.<n> SSR yields reliable and consistent improvements over direct solving, in-context learning, and single-source guidance.
  arXiv  Detail & Related papers  (2026-02-26T03:34:23Z)
• AutoDAN-Reasoning: Enhancing Strategies Exploration based Jailbreak Attacks with Test-Time Scaling [54.47844626555395]
  AutoDAN-Turbo employs a lifelong learning agent to build a rich library of attack strategies from scratch.<n>While highly effective, its test-time generation process involves sampling a strategy and generating a single corresponding attack prompt.<n>We propose to further improve the attack performance of AutoDAN-Turbo through test-time scaling.
  arXiv  Detail & Related papers  (2025-10-06T21:16:09Z)
• Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs [49.995906301946]
  Existing methods usually leverage a fixed strategy to guide Large Language Models (LLMs) to perform mathematical reasoning.<n>Our analysis reveals that the single strategy cannot adapt to problem-specific requirements and thus overlooks the trade-off between effectiveness and efficiency.<n>We propose Planning and Routing through Instance-Specific Modeling (PRISM), a novel framework that decouples mathematical reasoning into two stages: strategy planning and targeted execution.
  arXiv  Detail & Related papers  (2025-09-29T07:22:41Z)
• MetaScale: Test-Time Scaling with Evolving Meta-Thoughts [51.35594569020857]
  Experimental results demonstrate that MetaScale consistently outperforms standard inference approaches.<n> METASCALE scales more effectively with increasing sampling budgets and produces more structured, expert-level responses.
  arXiv  Detail & Related papers  (2025-03-17T17:59:54Z)
• LLM Strategic Reasoning: Agentic Study through Behavioral Game Theory [7.8900549152197215]
  We introduce an evaluation framework grounded in behavioral game theory, disentangling reasoning capability from contextual effects.<n>Testing 22 state-of-the-art LLMs, we find that GPT-o3-mini, GPT-o1, and DeepSeek-R1 dominate most games yet also demonstrate that the model scale alone does not determine performance.<n>In terms of prompting enhancement, Chain-of-Thought (CoT) prompting is not universally effective, as it increases strategic reasoning only for models at certain levels while providing limited gains elsewhere.
  arXiv  Detail & Related papers  (2025-02-27T18:58:31Z)
• StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving [76.5322280307861]
  StrategyLLM allows LLMs to perform inductive reasoning, deriving general strategies from specific task instances, and deductive reasoning, applying these general strategies to particular task examples, for constructing generalizable and consistent few-shot prompts. Experimental results demonstrate that StrategyLLM outperforms the competitive baseline CoT-SC that requires human-annotated solutions on 13 datasets across 4 challenging tasks without human involvement, including math reasoning (34.2% $rightarrow$ 38.8%), commonsense reasoning (70.3% $rightarrow$ 72.5%), algorithmic reasoning (73.7% $rightarrow$ 85.0
  arXiv  Detail & Related papers  (2023-11-15T09:18:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.