Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought

• URL: http://arxiv.org/abs/2604.22709v2
• Date: Mon, 27 Apr 2026 06:21:25 GMT
• Title: Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought
• Abstract summary: Abstract-CoT achieves up to $11.6times$ fewer reasoning tokens while demonstrating comparable performance across mathematical reasoning, instruction-following, and multi-hop reasoning.<n>Our findings highlight the potential for post-training latent reasoning mechanisms that enable efficient inference through a learned abstract reasoning language.
• Score: 13.72996269125319
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While long, explicit chains-of-thought (CoT) have proven effective on complex reasoning tasks, they are costly to generate during inference. Non-verbal reasoning methods have emerged with shorter generation lengths by leveraging continuous representations, yet their performance lags behind verbalized CoT. We propose $\textbf{Abstract Chain-of-Thought}$, a discrete latent reasoning post-training mechanism in which the language model produces a short sequence of tokens from a reserved vocabulary in lieu of a natural language CoT, before generating a response. To make previously unseen ''abstract'' tokens useful, we introduce a policy iteration-style warm-up loop that alternates between (i.) bottlenecking from a verbal CoT via masking and performing supervised fine-tuning, and (ii.) self-distillation by training the model to generate abstract tokens from the prompt alone via constrained decoding with the codebook. After warm-up, we optimize the generation of abstract sequences with warm-started reinforcement learning under constrained decoding. Abstract-CoT achieves up to $11.6\times$ fewer reasoning tokens while demonstrating comparable performance across mathematical reasoning, instruction-following, and multi-hop reasoning, and generalizes across language model families. We also find an emergent power law distribution over the abstract vocabulary, akin to those seen in natural language, that evolves across the training phases. Our findings highlight the potential for post-training latent reasoning mechanisms that enable efficient inference through a learned abstract reasoning language.

Related papers

• Efficient Reasoning via Thought Compression for Language Segmentation [28.138192925569317]
  Chain-of-thought (CoT) reasoning has significantly improved the performance of large multimodal models in language-guided segmentation.<n>We introduce WISE, a novel paradigm for efficient reasoning guided by the principle of textitthinking twice -- once for learning, once for speed.<n>We show WISE achieves state-of-the-art zero-shot performance on the ReasonSeg benchmark with 58.3 cIoU, while reducing the average reasoning length by nearly textbf5$times$ -- from 112 to just 23 tokens.
  arXiv  Detail & Related papers  (2026-04-02T13:45:01Z)
• CoLT: Reasoning with Chain of Latent Tool Calls [31.228763375347608]
  Chain-of-Thought (CoT) is a critical technique in enhancing the reasoning ability of Large Language Models (LLMs)<n>We propose CoLT, a novel framework that implements latent reasoning as tool calls''
  arXiv  Detail & Related papers  (2026-02-04T06:12:53Z)
• ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model [34.90582960625524]
  Long chains of thought (CoT) into compact latent tokens is crucial for efficient reasoning with large language models (LLMs)<n>We propose ImgCoT that replaces the reconstruction target from textual CoT to the visual CoT obtained by rendering CoT into images.<n>This substitutes linguistic bias with spatial inductive bias, enabling latent tokens to better capture global reasoning structure.
  arXiv  Detail & Related papers  (2026-01-30T09:06:45Z)
• Reflection Pretraining Enables Token-Level Self-Correction in Biological Sequence Models [82.79223371188756]
  Chain-of-Thought (CoT) prompting has advanced task-solving capabilities in natural language processing with large language models.<n>Applying CoT to non-natural language domains, such as protein and RNA language models, is not yet possible.<n>We introduce pretraining, for the first time in a biological sequence model, which enables the model to engage in intermediate reasoning.
  arXiv  Detail & Related papers  (2025-12-24T05:25:17Z)
• Think Clearly: Improving Reasoning via Redundant Token Pruning [57.01254508252785]
  We show that deliberately removing redundancy in the reasoning process significantly improves performance.<n>We demonstrate that our method significantly improves overall accuracy across reasoning-intensive benchmarks without any training.
  arXiv  Detail & Related papers  (2025-06-17T06:04:01Z)
• Latent Reasoning via Sentence Embedding Prediction [41.06552269338159]
  We present a framework that adapts a pretrained token-level LM to operate in sentence space by autoregressively predicting continuous embeddings of next sentences.<n>Our results indicate that pretrained LMs can effectively transition to abstract, structured reasoning within latent embedding spaces.
  arXiv  Detail & Related papers  (2025-05-28T10:28:35Z)
• Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space [62.54887038032942]
  We introduce Soft Thinking, a training-free method that emulates human-like "soft" reasoning by generating soft, abstract concept tokens.<n>These concept tokens are created by the probability-weighted mixture of token embeddings, which form the continuous concept space.<n>In essence, each generated concept token encapsulates multiple meanings from related discrete tokens, implicitly exploring various reasoning paths to converge.
  arXiv  Detail & Related papers  (2025-05-21T17:29:15Z)
• Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching [64.74765550805024]
  Chain-of-Thought prompting elicits step-by-step problem solving, but often at the cost of excessive verbosity in intermediate outputs.<n>We propose Sketch-of-Thought (SoT), a prompting framework that integrates cognitively inspired reasoning paradigms with linguistic constraints.<n>SoT achieves token reductions of up to 84% with minimal accuracy loss across 18 reasoning datasets.
  arXiv  Detail & Related papers  (2025-03-07T06:57:17Z)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning [53.57895922042783]
  Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data.<n>We propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens.
  arXiv  Detail & Related papers  (2025-02-05T15:33:00Z)
• Training Large Language Models to Reason in a Continuous Latent Space [71.0274000348354]
  We introduce a new paradigm called Coconut (Chain of Continuous Thought) to explore the potential of reasoning beyond language.<n>Instead of decoding this state into words, we feed it back to the model as the next input embedding directly in the continuous space.<n>This latent reasoning paradigm enables an advanced reasoning pattern, where continuous thoughts can encode multiple alternative next steps.
  arXiv  Detail & Related papers  (2024-12-09T18:55:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.