Fast Thinking for Large Language Models

• URL: http://arxiv.org/abs/2509.23633v1
• Date: Sun, 28 Sep 2025 04:19:48 GMT
• Title: Fast Thinking for Large Language Models
• Abstract summary: We introduce Latent Codebooks for Fast Thinking, a framework that uses concise CoT sketches only during training to learn a codebook of discrete strategy priors.<n>At inference, the model conditions on a handful of continuous thinking switches distilled from the codebook in a single pass, enabling strategy-level guidance without producing explicit reasoning tokens.
• Score: 67.7238685892317
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning-oriented Large Language Models (LLMs) often rely on generating explicit tokens step by step, and their effectiveness typically hinges on large-scale supervised fine-tuning or reinforcement learning. While Chain-of-Thought (CoT) techniques substantially enhance performance on complex reasoning tasks, they remain inefficient, requiring long reasoning traces that increase latency and token usage. In this work, we introduce Latent Codebooks for Fast Thinking, a framework that uses concise CoT sketches only during training to learn a codebook of discrete strategy priors. At inference, the model conditions on a handful of continuous thinking vectors distilled from the codebook in a single pass, enabling strategy-level guidance without producing explicit reasoning tokens. To complement this design, we propose GainRouter, a lightweight routing mechanism that adaptively switches between fast codebook guided inference and slow explicit reasoning, thereby suppressing overthinking and reducing unnecessary token generation. Experiments across multiple reasoning benchmarks show that our approach achieves competitive or superior accuracy while substantially lowering inference cost, offering a practical path toward efficient and controllable reasoning in large language models.

Related papers

• ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning [48.3829293800053]
  We propose ChainPrune, a novel reasoning path semantic structural optimization method.<n>We consolidate self-generated reasoning paths into a tree-based structure, followed by a multi-criteria dominant path selection process.<n>We incorporate a DPO-based preference learning method combined with supervised loss, effectively mitigating false reward suppression.
  arXiv  Detail & Related papers  (2026-08-22T09:09:05Z)
• Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs [44.74237674161132]
  Reasoning in Multimodal Large Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction.<n>Explicit text-based Chain-of-Thought (CoT) is computationally expensive and prone to visual hallucinations.<n>Existing latent reasoning methods typically require costly training.
  arXiv  Detail & Related papers  (2026-08-04T10:46:10Z)
• CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts [25.917787556467776]
  Chain-of-thought (CoT) reasoning has enabled multi-modal large language models (MLLMs) to tackle complex visual reasoning tasks.<n>We propose CoLT, a novel framework that teaches multi-modal models to reason through a chain of latent thought representations.<n>Compared to text CoT methods, CoLT can notably reduce the inference time by 10.1$times$ and text decoding time by 22.6$times$.
  arXiv  Detail & Related papers  (2026-06-30T17:24:40Z)
• CARE: Competence-Aware Reward Shaping for Adaptive Reasoning Length in Video-MLLMs [50.189987475377656]
  We propose CARE, a competence-aware reward shaping framework for adaptive reasoning length optimization in multimodal reasoning.<n> CARE maintains a smoothed competence estimate via an exponential moving average of pass rates, and uses it to route training into progressive stages.<n>Experiments on multiple video reasoning and general video understanding benchmarks demonstrate that CARE consistently improves reasoning accuracy, stabilizes reinforcement learning, and significantly enhances token efficiency.
  arXiv  Detail & Related papers  (2026-06-18T08:28:26Z)
• LogitsCoder: Towards Efficient Chain-of-Thought Path Search via Logits Preference Decoding for Code Generation [86.08600027874662]
  We propose LogitsCoder, a novel framework that enhances chain-of-thought reasoning through lightweight, logit-level control mechanisms for code generation.<n>We show that LogitsCoder produces more efficient and higher-quality reasoning chains, leading to superior code generation performance compared to baseline methods.
  arXiv  Detail & Related papers  (2026-02-15T08:52:19Z)
• CoLT: Reasoning with Chain of Latent Tool Calls [31.228763375347608]
  Chain-of-Thought (CoT) is a critical technique in enhancing the reasoning ability of Large Language Models (LLMs)<n>We propose CoLT, a novel framework that implements latent reasoning as tool calls''
  arXiv  Detail & Related papers  (2026-02-04T06:12:53Z)
• Debiasing Large Language Models via Adaptive Causal Prompting with Sketch-of-Thought [18.725256563820952]
  We propose an Adaptive Causal Prompting with Sketch-of-Thought (ACPS) framework.<n>ACPS replaces verbose Chain-of-Thought (CoT) with concise Sketch-of-Thought.<n>ACPS consistently outperforms existing prompting baselines in terms of accuracy, robustness, and computational efficiency.
  arXiv  Detail & Related papers  (2026-01-13T00:58:43Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generation [74.37307916314407]
  We propose a framework dubbed ConciseHint, which continuously encourages the reasoning model to speak concisely.<n>Experiments on the state-of-the-art LRMs, including DeepSeek-R1 and Qwen-3 series, demonstrate that our method can effectively produce concise reasoning.
  arXiv  Detail & Related papers  (2025-06-23T16:20:44Z)
• PixelThink: Towards Efficient Chain-of-Pixel Reasoning [70.32510083790069]
  PixelThink is a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty.<n>It learns to compress reasoning length in accordance with scene complexity and predictive confidence.<n> Experimental results demonstrate that the proposed approach improves both reasoning efficiency and overall segmentation performance.
  arXiv  Detail & Related papers  (2025-05-29T17:55:49Z)
• Adaptive Deep Reasoning: Triggering Deep Thinking When Needed [28.575411507835973]
  Large language models (LLMs) have shown impressive capabilities in handling complex tasks through long-chain reasoning.<n>We propose a novel approach that autonomously switches between short and long-chain reasoning chains based on problem complexity.<n>This advancement enhances the practicality of reasoning in large language models for real-world applications.
  arXiv  Detail & Related papers  (2025-05-26T15:08:51Z)
• Efficient Reasoning Models: A Survey [73.00621058885054]
  This survey aims to provide a comprehensive overview of recent advances in efficient reasoning.<n>It categorizes existing works into three key directions: (1) shorter - compressing lengthy CoTs into concise yet effective reasoning chains; (2) smaller - developing compact language models with strong reasoning capabilities; and (3) faster.
  arXiv  Detail & Related papers  (2025-04-15T06:28:00Z)
• Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching [64.74765550805024]
  Chain-of-Thought prompting elicits step-by-step problem solving, but often at the cost of excessive verbosity in intermediate outputs.<n>We propose Sketch-of-Thought (SoT), a prompting framework that integrates cognitively inspired reasoning paradigms with linguistic constraints.<n>SoT achieves token reductions of up to 84% with minimal accuracy loss across 18 reasoning datasets.
  arXiv  Detail & Related papers  (2025-03-07T06:57:17Z)
• Efficient Reasoning with Hidden Thinking [48.96945580741641]
  Chain-of-Thought (CoT) reasoning has become a powerful framework for improving complex problem-solving capabilities.<n>We propose $textbfHeima$ (as hidden llama), an efficient reasoning framework that leverages reasoning CoTs at hidden latent space.<n>Heima model achieves higher generation efficiency while maintaining or even better zero-shot task accuracy.
  arXiv  Detail & Related papers  (2025-01-31T15:10:29Z)
• Think Beyond Size: Adaptive Prompting for More Effective Reasoning [0.0]
  We introduce Adaptive Prompting, a dynamic and iterative framework designed to enhance reasoning by incorporating real-time adjustments to prompt structures and validation mechanisms.<n>Results demonstrate that Adaptive Prompting significantly improves performance on diverse reasoning benchmarks, including arithmetic reasoning (GSM8K, MultiArithm), logical reasoning and commonsense tasks.<n>Our approach enables smaller models to achieve competitive performance with larger counterparts, such as GPT-4, while maintaining computational efficiency.
  arXiv  Detail & Related papers  (2024-10-10T17:14:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.