Thinking with Reasoning Skills: Fewer Tokens, More Accuracy

• URL: http://arxiv.org/abs/2604.21764v2
• Date: Sat, 25 Apr 2026 16:45:22 GMT
• Title: Thinking with Reasoning Skills: Fewer Tokens, More Accuracy
• Abstract summary: Reasoning LLMs often spend substantial tokens on long intermediate reasoning traces when solving new problems.<n>We propose to summarize and store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration.
• Score: 10.06085033727538
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reasoning LLMs often spend substantial tokens on long intermediate reasoning traces (e.g., chain-of-thought) when solving new problems. We propose to summarize and store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration, and to retrieve these skills at inference time to guide future reasoning. Unlike the prevailing \emph{reasoning from scratch} paradigm, our approach first recalls relevant skills for each query, helping the model avoid redundant detours and focus on effective solution paths. We evaluate our method on coding and mathematical reasoning tasks, and find that it significantly reduces reasoning tokens while improving overall performance. The resulting lower per-request cost indicates strong practical and economic potential for real-world deployment.

Related papers

• Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning [9.687158786222994]
  Large reasoning models (LRMs) improve problem solving through extended reasoning, but often misallocate test-time compute.<n>We propose Budget-Efficient Thinking (BET), a two-stage framework that combines behavioral cold-start with GRPO under an investment-cost-aware reward.<n>BET reduces reasoning tokens by 55% on average while achieving overall performance improvements.
  arXiv  Detail & Related papers  (2026-05-12T06:51:31Z)
• Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs [46.272771457924186]
  We propose textbfDraft-Thinking, which guides models to first learn a concise textitdraft-style reasoning structure that retains only the critical reasoning steps.<n>Experiments demonstrate that Draft-Thinking substantially reduces reasoning budget while largely preserving reasoning performance.
  arXiv  Detail & Related papers  (2026-02-28T09:57:52Z)
• In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback [38.915062716409686]
  InTRO is a new framework that enables both token-level exploration and self-feedback for accurate and concise reasoning.<n>InTRO consistently outperforms other baselines, raising solution accuracy by up to 20% relative to the base model.<n>Its chains of thought are notably more concise, exhibiting reduced verbosity.
  arXiv  Detail & Related papers  (2025-11-13T01:47:06Z)
• Learning to Reason Efficiently with Discounted Reinforcement Learning [57.28725125116862]
  Large reasoning models (LRMs) often consume excessive tokens, inflating computational cost and latency.<n>We challenge the assumption that longer responses improve accuracy.<n>By penalizing reasoning tokens using a discounted reinforcement learning setup (interpretable as a small token cost) and analyzing Blackwell optimality in restricted policy classes, we encourage concise yet accurate reasoning.
  arXiv  Detail & Related papers  (2025-10-27T16:17:45Z)
• Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model [7.8354921036790275]
  Large Reasoning Models (LRMs) excel at solving complex problems but face an overthinking dilemma.<n>When handling simple tasks, they often produce verbose responses overloaded with thinking tokens.<n>These tokens trigger unnecessary high-level reasoning behaviors like reflection and backtracking, reducing efficiency.
  arXiv  Detail & Related papers  (2025-06-30T13:30:33Z)
• Think Clearly: Improving Reasoning via Redundant Token Pruning [57.01254508252785]
  We show that deliberately removing redundancy in the reasoning process significantly improves performance.<n>We demonstrate that our method significantly improves overall accuracy across reasoning-intensive benchmarks without any training.
  arXiv  Detail & Related papers  (2025-06-17T06:04:01Z)
• Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models [51.85792055455284]
  Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to perform complex reasoning tasks.<n>System 1 reasoning is computationally efficient but leads to suboptimal performance.<n>System 2 reasoning often incurs substantial computational costs due to its slow thinking nature and inefficient or unnecessary reasoning behaviors.
  arXiv  Detail & Related papers  (2025-03-31T17:58:07Z)
• Efficient Inference for Large Reasoning Models: A Survey [74.17203483365171]
  Large Reasoning Models (LRMs) significantly improve the reasoning ability of Large Language Models (LLMs) by learning to reason.<n>However, their deliberative reasoning process leads to inefficiencies in token usage, memory consumption, and inference time.<n>This survey provides a review of efficient inference methods designed specifically for LRMs, focusing on mitigating token inefficiency while preserving the reasoning quality.
  arXiv  Detail & Related papers  (2025-03-29T13:27:46Z)
• A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond [88.5807076505261]
  Large Reasoning Models (LRMs) have demonstrated strong performance gains by scaling up the length of Chain-of-Thought (CoT) reasoning during inference.<n>A growing concern lies in their tendency to produce excessively long reasoning traces.<n>This inefficiency introduces significant challenges for training, inference, and real-world deployment.
  arXiv  Detail & Related papers  (2025-03-27T15:36:30Z)
• FINEREASON: Evaluating and Improving LLMs' Deliberate Reasoning through Reflective Puzzle Solving [90.88021670297664]
  FINEREASON is a logic-puzzle benchmark for evaluation of large language models' reasoning capabilities.<n>We introduce two tasks: state checking, and state transition, for a comprehensive evaluation of how models assess the current situation and plan the next move.<n>We show that models trained on our state checking and transition data demonstrate gains in math reasoning by up to 5.1% on GSM8K.
  arXiv  Detail & Related papers  (2025-02-27T16:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.