A Model with No Head and Many Thoughts

• URL: http://arxiv.org/abs/2608.31069v1
• Date: Mon, 31 Aug 2026 16:45:41 GMT
• Title: A Model with No Head and Many Thoughts
• Abstract summary: We introduce Soft Latent Thinking, a method that replaces the LM head during reasoning with a lightweight projector.<n>We show that Soft Latent Thinking consistently improves pass@k across all k while reducing per-step compute during chain-of-thought.
• Score: 8.93920041624494
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models decode by projecting hidden states through a large vocabulary head at every step. This operation is computationally costly and forces all reasoning to be expressed in discrete tokens. We introduce Soft Latent Thinking, a method that replaces the LM head during reasoning with a lightweight projector, enabling autoregressive rollout in embedding space where reasoning steps remain continuous rather than tokenized. Experiments on DeepSeek-Qwen-1.5B and LLaMA-3.2-3B show that Soft Latent Thinking consistently improves pass@k across all k while reducing per-step compute during chain-of-thought. Our method achieves the highest pass@32 among all soft-thinking approaches, demonstrating that effective reasoning can be carried out in continuous space without discrete token generation.

Related papers

• Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs [16.72007494160683]
  Chain-of-thought reasoning unfolds in discrete token space.<n> Reasoning instead in a model's continuous representation space.<n>We instantiate this as Latent Recurrent Thoughts.
  arXiv  Detail & Related papers  (2026-09-01T11:56:17Z)
• CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts [25.917787556467776]
  Chain-of-thought (CoT) reasoning has enabled multi-modal large language models (MLLMs) to tackle complex visual reasoning tasks.<n>We propose CoLT, a novel framework that teaches multi-modal models to reason through a chain of latent thought representations.<n>Compared to text CoT methods, CoLT can notably reduce the inference time by 10.1$times$ and text decoding time by 22.6$times$.
  arXiv  Detail & Related papers  (2026-06-30T17:24:40Z)
• SPOT: Span-level Pause-of-Thought for Efficient and Interpretable Latent Reasoning in Large Language Models [15.95627037350657]
  Explicit Chain-of-Thought (CoT) computation incurs high inference cost due to internalize token-level traces.<n>We propose SPOT, a flexible framework that compresses explicit CoT into compact latent pause tokens without enforcing a fixed response template.<n>Experiments on reasoning benchmarks demonstrate that SPOT improves accuracy by 2.3 points on average while reducing generated tokens by 37.5%.
  arXiv  Detail & Related papers  (2026-03-06T12:34:27Z)
• Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning [62.680551162054975]
  We introduce an end-to-end framework where LLMs learn to self-regulate the granularity of the reasoning steps through dynamic summarization.<n>We apply reinforcement learning to incentivize this capability further, uncovering a critical insight: the accuracy gap between the highly efficient Fold mode and the exhaustive Unfold mode progressively narrows.<n>Our Accordion-Thinker demonstrates that with learned self-compression, LLMs can tackle complex reasoning tasks with minimal dependency token overhead.
  arXiv  Detail & Related papers  (2026-02-03T08:34:20Z)
• Latent Reasoning in LLMs as a Vocabulary-Space Superposition [80.01651003144282]
  Large language models (LLMs) demonstrate strong reasoning abilities with chain-of-thought prompting, but explicit reasoning introduces substantial computational overhead.<n>Recent work on latent reasoning reduces this cost by reasoning in latent space without explicit supervision, but performance drops significantly.<n>To address this, we restrict the latent space to the column space of the LLM vocabulary, treating latent reasoning as a superposition over vocabulary probabilities.<n>Once latent reasoning concludes, it collapses into an eigenstate of explicit reasoning to yield the final answer.<n>Latent-SFT sets a new state of the art on GSM8k, matching explicit
  arXiv  Detail & Related papers  (2025-10-17T10:51:20Z)
• MARCOS: Deep Thinking by Markov Chain of Continuous Thoughts [82.46857666702924]
  We present a new paradigm for reasoning in large language models (LLMs)<n>Instead of autoregressively generating tokens, we model reasoning as a hidden Markov chain of continuous, high-dimensional "thoughts"<n>For the first time, MARCOS achieves performance comparable to token-based CoT, even surpassing it by 4.7% on GSM8K with up to 15.7x speedup in inference.
  arXiv  Detail & Related papers  (2025-09-29T16:44:22Z)
• The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models [54.88805865447848]
  We show that instruct models achieve higher efficiency overall, and problem difficulty affects efficiency.<n>We propose COTHINK, a simple two-stage pipeline: an instruct model drafts a brief outline, and a thinking model expands it.<n>On GSM8K, MATH500, and AIME24, COTHINK cuts token usage by 21.1% while keeping accuracy on four thinking models, and remains competitive with strong efficiency baselines.
  arXiv  Detail & Related papers  (2025-05-28T06:24:45Z)
• Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space [62.54887038032942]
  We introduce Soft Thinking, a training-free method that emulates human-like "soft" reasoning by generating soft, abstract concept tokens.<n>These concept tokens are created by the probability-weighted mixture of token embeddings, which form the continuous concept space.<n>In essence, each generated concept token encapsulates multiple meanings from related discrete tokens, implicitly exploring various reasoning paths to converge.
  arXiv  Detail & Related papers  (2025-05-21T17:29:15Z)
• Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching [64.74765550805024]
  Chain-of-Thought prompting elicits step-by-step problem solving, but often at the cost of excessive verbosity in intermediate outputs.<n>We propose Sketch-of-Thought (SoT), a prompting framework that integrates cognitively inspired reasoning paradigms with linguistic constraints.<n>SoT achieves token reductions of up to 84% with minimal accuracy loss across 18 reasoning datasets.
  arXiv  Detail & Related papers  (2025-03-07T06:57:17Z)
• LightThinker: Thinking Step-by-Step Compression [74.34839026338342]
  We propose LightThinker, a method that enables large language models to dynamically compress intermediate thoughts during reasoning.<n>Inspired by human cognitive processes, LightThinker compresses thought steps into compact representations and discards the original reasoning chains.<n>Experiments show that LightThinker reduces peak memory usage and inference time, while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-02-21T16:57:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.