R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning

• URL: http://arxiv.org/abs/2509.22131v2
• Date: Mon, 29 Sep 2025 03:19:32 GMT
• Title: R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
• Abstract summary: Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales.<n>We propose the Reasoning Capsule (R-Capsule), a framework that aims to combine the efficiency of latent reasoning with the transparency of explicit CoT.
• Score: 25.87953249848607
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales. However, CoT's verbosity increases latency and memory usage and may propagate early errors across long chains. We propose the Reasoning Capsule (R-Capsule), a framework that aims to combine the efficiency of latent reasoning with the transparency of explicit CoT. The core idea is to compress the high-level plan into a small set of learned latent tokens (a Reasoning Capsule) while keeping execution steps lightweight or explicit. This hybrid approach is inspired by the Information Bottleneck (IB) principle, where we encourage the capsule to be approximately minimal yet sufficient for the task. Minimality is encouraged via a low-capacity bottleneck, which helps improve efficiency. Sufficiency is encouraged via a dual objective: a primary task loss for answer accuracy and an auxiliary plan-reconstruction loss that encourages the capsule to faithfully represent the original textual plan. The reconstruction objective helps ground the latent space, thereby improving interpretability and reducing the use of uninformative shortcuts. Our framework strikes a balance between efficiency, accuracy, and interpretability, thereby reducing the visible token footprint of reasoning while maintaining or improving accuracy on complex benchmarks. Our codes are available at: https://anonymous.4open.science/r/Reasoning-Capsule-7BE0

Related papers

• ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning [69.64972562984882]
  We propose ThoughtFold, a framework that leverages fine-grained preference learning to redundant explorations for efficient reasoning.<n>We show that ThoughtFold significantly enhances efficiency.<n>It reduces the token usage of DeepSeek-R1-Distill-Qwen-7B by approximately 56% while maintaining state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2026-06-02T11:21:27Z)
• Efficient Reasoning via Thought Compression for Language Segmentation [28.138192925569317]
  Chain-of-thought (CoT) reasoning has significantly improved the performance of large multimodal models in language-guided segmentation.<n>We introduce WISE, a novel paradigm for efficient reasoning guided by the principle of textitthinking twice -- once for learning, once for speed.<n>We show WISE achieves state-of-the-art zero-shot performance on the ReasonSeg benchmark with 58.3 cIoU, while reducing the average reasoning length by nearly textbf5$times$ -- from 112 to just 23 tokens.
  arXiv  Detail & Related papers  (2026-04-02T13:45:01Z)
• Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck [12.360124156284305]
  Existing "Budget Forcing" methods suppress both essential reasoning and redundant filler.<n>We recast efficient reasoning as a lossy compression problem under the Information Bottleneck (IB) principle.<n>In contrast to naive token-counting-based approaches, we introduce a semantic prior that measures token cost by surprisal under a language model prior.
  arXiv  Detail & Related papers  (2026-03-09T14:56:57Z)
• CoLT: Reasoning with Chain of Latent Tool Calls [31.228763375347608]
  Chain-of-Thought (CoT) is a critical technique in enhancing the reasoning ability of Large Language Models (LLMs)<n>We propose CoLT, a novel framework that implements latent reasoning as tool calls''
  arXiv  Detail & Related papers  (2026-02-04T06:12:53Z)
• Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning [62.680551162054975]
  We introduce an end-to-end framework where LLMs learn to self-regulate the granularity of the reasoning steps through dynamic summarization.<n>We apply reinforcement learning to incentivize this capability further, uncovering a critical insight: the accuracy gap between the highly efficient Fold mode and the exhaustive Unfold mode progressively narrows.<n>Our Accordion-Thinker demonstrates that with learned self-compression, LLMs can tackle complex reasoning tasks with minimal dependency token overhead.
  arXiv  Detail & Related papers  (2026-02-03T08:34:20Z)
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs [57.217593337454026]
  TokenSqueeze is a novel Long2Short method that condenses reasoning paths while preserving performance and relying exclusively on self-generated data.<n>We show that TokenSqueeze reduces token usage while maintaining accuracy on the MATH500 benchmark.
  arXiv  Detail & Related papers  (2025-11-17T10:38:56Z)
• Fast Thinking for Large Language Models [67.7238685892317]
  We introduce Latent Codebooks for Fast Thinking, a framework that uses concise CoT sketches only during training to learn a codebook of discrete strategy priors.<n>At inference, the model conditions on a handful of continuous thinking switches distilled from the codebook in a single pass, enabling strategy-level guidance without producing explicit reasoning tokens.
  arXiv  Detail & Related papers  (2025-09-28T04:19:48Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generation [74.37307916314407]
  We propose a framework dubbed ConciseHint, which continuously encourages the reasoning model to speak concisely.<n>Experiments on the state-of-the-art LRMs, including DeepSeek-R1 and Qwen-3 series, demonstrate that our method can effectively produce concise reasoning.
  arXiv  Detail & Related papers  (2025-06-23T16:20:44Z)
• ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancy [8.962703809086628]
  ThinkLess is an inference-efficient framework that terminates reasoning generation early and maintains output quality without modifying the model.<n>We show that ThinkLess achieves comparable accuracy to full-length Chain-of-Thought (CoT) decoding while greatly reducing decoding time and memory consumption.
  arXiv  Detail & Related papers  (2025-05-21T15:58:16Z)
• Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning [15.137717200618454]
  Prune-on-Logic is a framework that transforms Long-CoT into logic graphs and selectively prunes low-utility reasoning steps.<n>We find that verification pruning consistently improves accuracy while reducing token usage, whereas reasoning or indiscriminate pruning degrades performance.
  arXiv  Detail & Related papers  (2025-05-20T16:38:32Z)
• Hawkeye:Efficient Reasoning with Model Collaboration [7.26791045376255]
  Chain-of-Thought (CoT) reasoning has demonstrated remarkable effectiveness in enhancing the reasoning abilities of large language models (LLMs)<n>Most CoT tokens are unnecessary, and retaining only a small portion of them is sufficient for producing high-quality responses.<n>We propose HAWKEYE, a novel post-training and inference framework where a large model produces concise CoT instructions to guide a smaller model in response generation.
  arXiv  Detail & Related papers  (2025-04-01T05:09:04Z)
• Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching [64.74765550805024]
  Chain-of-Thought prompting elicits step-by-step problem solving, but often at the cost of excessive verbosity in intermediate outputs.<n>We propose Sketch-of-Thought (SoT), a prompting framework that integrates cognitively inspired reasoning paradigms with linguistic constraints.<n>SoT achieves token reductions of up to 84% with minimal accuracy loss across 18 reasoning datasets.
  arXiv  Detail & Related papers  (2025-03-07T06:57:17Z)
• Chain of Draft: Thinking Faster by Writing Less [37.492654173517046]
  Chain of Draft (CoD) is a novel paradigm inspired by human cognitive processes.<n>CoD generates minimalistic yet informative intermediate reasoning outputs while solving tasks.
  arXiv  Detail & Related papers  (2025-02-25T19:36:06Z)
• Efficient Reasoning with Hidden Thinking [48.96945580741641]
  Chain-of-Thought (CoT) reasoning has become a powerful framework for improving complex problem-solving capabilities.<n>We propose $textbfHeima$ (as hidden llama), an efficient reasoning framework that leverages reasoning CoTs at hidden latent space.<n>Heima model achieves higher generation efficiency while maintaining or even better zero-shot task accuracy.
  arXiv  Detail & Related papers  (2025-01-31T15:10:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.