CoLT: Reasoning with Chain of Latent Tool Calls

• URL: http://arxiv.org/abs/2602.04246v1
• Date: Wed, 04 Feb 2026 06:12:53 GMT
• Title: CoLT: Reasoning with Chain of Latent Tool Calls
• Authors: Fangwei Zhu, Zhifang Sui,
• Abstract summary: Chain-of-Thought (CoT) is a critical technique in enhancing the reasoning ability of Large Language Models (LLMs)<n>We propose CoLT, a novel framework that implements latent reasoning as tool calls''
• Score: 31.228763375347608
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Chain-of-Thought (CoT) is a critical technique in enhancing the reasoning ability of Large Language Models (LLMs), and latent reasoning methods have been proposed to accelerate the inefficient token-level reasoning chain. We notice that existing latent reasoning methods generally require model structure augmentation and exhaustive training, limiting their broader applicability. In this paper, we propose CoLT, a novel framework that implements latent reasoning as ``tool calls''. Instead of reasoning entirely in the latent space, CoLT generates seed tokens that contain information of a reasoning step. When a latent tool call is triggered, a smaller external model will take the hidden states of seed tokens as its input, and unpack the seed tokens back to a full reasoning step. In this way, we can ensure that the main model reasons in the explicit token space, preserving its ability while improving efficiency. Experimental results on four mathematical datasets demonstrate that CoLT achieves higher accuracy and shorter reasoning length than baseline latent models, and is compatible with reinforcement learning algorithms and different decoder structures.

Related papers

• Chain Of Thought Compression: A Theoritical Analysis [24.613200477865572]
  Chain-of-Thought (CoT) has unlocked advanced reasoning abilities of Large Language Models.<n>CoT incurs prohibitive computational costs due to generation of extra tokens.<n>Recent studies show that compressing reasoning steps into latent states, or implicit CoT compression, offers a token-efficient alternative.
  arXiv  Detail & Related papers  (2026-01-29T11:42:03Z)
• Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization [9.193078163792427]
  Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems.<n>Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states.<n>We introduce PLaT, a framework that reformulates latent reasoning as planning by fundamentally decouple reasoning from verbalization.
  arXiv  Detail & Related papers  (2026-01-29T07:38:18Z)
• Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models [39.5490415037017]
  Chain-of-Thought (CoT) prompting has improved the reasoning performance of large language models (LLMs)<n>It remains unclear why it works and whether it is the unique mechanism for triggering reasoning in large language models.
  arXiv  Detail & Related papers  (2026-01-12T23:01:21Z)
• Latent Reasoning in LLMs as a Vocabulary-Space Superposition [80.01651003144282]
  Large language models (LLMs) demonstrate strong reasoning abilities with chain-of-thought prompting, but explicit reasoning introduces substantial computational overhead.<n>Recent work on latent reasoning reduces this cost by reasoning in latent space without explicit supervision, but performance drops significantly.<n>To address this, we restrict the latent space to the column space of the LLM vocabulary, treating latent reasoning as a superposition over vocabulary probabilities.<n>Once latent reasoning concludes, it collapses into an eigenstate of explicit reasoning to yield the final answer.<n>Latent-SFT sets a new state of the art on GSM8k, matching explicit
  arXiv  Detail & Related papers  (2025-10-17T10:51:20Z)
• Fast Thinking for Large Language Models [67.7238685892317]
  We introduce Latent Codebooks for Fast Thinking, a framework that uses concise CoT sketches only during training to learn a codebook of discrete strategy priors.<n>At inference, the model conditions on a handful of continuous thinking switches distilled from the codebook in a single pass, enabling strategy-level guidance without producing explicit reasoning tokens.
  arXiv  Detail & Related papers  (2025-09-28T04:19:48Z)
• A Survey on Latent Reasoning [100.54120559169735]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities.<n>CoT reasoning that verbalizes intermediate steps limits the model's expressive bandwidth.<n>Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state.
  arXiv  Detail & Related papers  (2025-07-08T17:29:07Z)
• Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer [0.8738725605667471]
  Chain-of-thought (CoT) reasoning has enabled transformer-based language models to excel at complex mathematics and multi-step planning.<n>In standard decoder-only architectures, these reasoning steps are externalized in natural language, improving interpretability at the cost of efficiency.<n>We investigate whether such reasoning structures emerge in Huginn-3.5B, a depth-recurrent Transformer that reuses layers at inference time without increasing parameter count.
  arXiv  Detail & Related papers  (2025-07-02T23:35:21Z)
• ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generation [74.37307916314407]
  We propose a framework dubbed ConciseHint, which continuously encourages the reasoning model to speak concisely.<n>Experiments on the state-of-the-art LRMs, including DeepSeek-R1 and Qwen-3 series, demonstrate that our method can effectively produce concise reasoning.
  arXiv  Detail & Related papers  (2025-06-23T16:20:44Z)
• Efficient Inference for Large Reasoning Models: A Survey [74.17203483365171]
  Large Reasoning Models (LRMs) significantly improve the reasoning ability of Large Language Models (LLMs) by learning to reason.<n>However, their deliberative reasoning process leads to inefficiencies in token usage, memory consumption, and inference time.<n>This survey provides a review of efficient inference methods designed specifically for LRMs, focusing on mitigating token inefficiency while preserving the reasoning quality.
  arXiv  Detail & Related papers  (2025-03-29T13:27:46Z)
• Efficient Reasoning with Hidden Thinking [48.96945580741641]
  Chain-of-Thought (CoT) reasoning has become a powerful framework for improving complex problem-solving capabilities.<n>We propose $textbfHeima$ (as hidden llama), an efficient reasoning framework that leverages reasoning CoTs at hidden latent space.<n>Heima model achieves higher generation efficiency while maintaining or even better zero-shot task accuracy.
  arXiv  Detail & Related papers  (2025-01-31T15:10:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.