Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models

• URL: http://arxiv.org/abs/2511.08577v1
• Date: Wed, 12 Nov 2025 02:05:42 GMT
• Title: Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models
• Abstract summary: Think-at-Hard (TaH) is a dynamic latent thinking method that iterates deeper only at hard tokens.<n>TaH boosts reasoning performance across five challenging benchmarks.
• Score: 22.525318796588568
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Improving reasoning capabilities of Large Language Models (LLMs), especially under parameter constraints, is crucial for real-world applications. Prior work proposes recurrent transformers, which allocate a fixed number of extra iterations per token to improve generation quality. After the first, standard forward pass, instead of verbalization, last-layer hidden states are fed back as inputs for additional iterations to refine token predictions. Yet we identify a latent overthinking phenomenon: easy token predictions that are already correct after the first pass are sometimes revised into errors in additional iterations. To address this, we propose Think-at-Hard (TaH), a dynamic latent thinking method that iterates deeper only at hard tokens. It employs a lightweight neural decider to trigger latent iterations only at tokens that are likely incorrect after the standard forward pass. During latent iterations, Low-Rank Adaptation (LoRA) modules shift the LLM objective from general next-token prediction to focused hard-token refinement. We further introduce a duo-causal attention mechanism that extends attention from the token sequence dimension to an additional iteration depth dimension. This enables cross-iteration information flow while maintaining full sequential parallelism. Experiments show that TaH boosts LLM reasoning performance across five challenging benchmarks while maintaining the same parameter count. Compared with baselines that iterate twice for all output tokens, TaH delivers 8.1-11.3% accuracy gains while exempting 94% of tokens from the second iteration. Against strong single-iteration Qwen3 models finetuned with the same data, it also delivers 4.0-5.0% accuracy gains. When allowing less than 3% additional parameters from LoRA and the iteration decider, the gains increase to 8.5-12.6% and 5.3-5.4%, respectively. Our code is available at https://github.com/thu-nics/TaH.

Related papers

• PARTREP: Learning What to Repeat for Decoder-only LLMs [12.053018926036678]
  PartRep is a selective augmentation method that appends only the most informative tokens.<n>We train a lightweight gate that predicts high-NLL tokens from early-layer hidden states.<n>PartRep retains most of the gains of full repetition while using only 59.4% of its KV cache and 79.0% of its prefill FLOPs.
  arXiv  Detail & Related papers  (2026-07-02T07:07:28Z)
• LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification [35.08680804423239]
  Chain-of-thought (CoT) reasoning improves large language models (LLMs) on difficult tasks, but it also makes inference expensive.<n>We propose Latent-Then-Explicit Reasoning (LaTER), a two-stage paradigm that first performs bounded exploration in a continuous latent space.<n>LaTER projects final-layer hidden states back to the input embedding space, preserves the latent KV cache, and uses entropy and model-native stop-token probes to decide when to switch.
  arXiv  Detail & Related papers  (2026-05-08T06:23:58Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing [3.861463840588074]
  Large language models (LLMs) exhibit latent multi-token prediction (MTP) capabilities despite being trained solely for next-token generation.<n>We propose a training-free MTP approach that probes an LLM using on-the-fly mask tokens drawn from its embedding space.<n>Our method constructs a speculative token tree by sampling top-K candidates from mask-token logits and applies a lightweight pruning strategy to retain high-probability continuations.
  arXiv  Detail & Related papers  (2026-03-18T17:14:01Z)
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs [57.217593337454026]
  TokenSqueeze is a novel Long2Short method that condenses reasoning paths while preserving performance and relying exclusively on self-generated data.<n>We show that TokenSqueeze reduces token usage while maintaining accuracy on the MATH500 benchmark.
  arXiv  Detail & Related papers  (2025-11-17T10:38:56Z)
• Continuous Autoregressive Language Models [56.49239051750678]
  We introduce Continuous Autoregressive Language Models (CALM)<n>CALM uses a high-fidelity autoencoder to compress a chunk of K tokens into a single continuous vector.<n>We develop a comprehensive likelihood-free framework that enables robust training, evaluation, and controllable sampling.
  arXiv  Detail & Related papers  (2025-10-31T17:58:11Z)
• Diffusion Language Models Know the Answer Before Decoding [56.96815863705218]
  Diffusion language models (DLMs) have emerged as an alternative to autoregressive approaches.<n>Our work highlights and leverage an overlooked property of DLMs early answer convergence.<n>We introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding.
  arXiv  Detail & Related papers  (2025-08-27T15:40:25Z)
• Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential [12.719829360337833]
  We propose a novel framework that leverages the inherent knowledge of vanilla autoregressive language models about future tokens.<n>Our method achieves significant speedups through supervised fine-tuning on pretrained models.
  arXiv  Detail & Related papers  (2025-07-16T02:31:40Z)
• Multipole Attention for Efficient Long Context Reasoning [64.94673641704289]
  Large Reasoning Models (LRMs) have shown promising accuracy improvements on complex problem-solving tasks.<n>LRMs need to generate long chain-of-thought reasoning in order to think before answering.<n>We introduce Multipole Attention, which accelerates autoregressive reasoning by only computing exact attention for the most important tokens.
  arXiv  Detail & Related papers  (2025-06-16T03:00:40Z)
• Fast Quiet-STaR: Thinking Without Thought Tokens [51.79231070632772]
  Fast Quiet STaR is a more efficient reasoning framework that preserves the benefits of token-level reasoning while reducing computational cost.<n>Our method introduces a curriculum learning based training strategy that gradually reduces the number of thought tokens.<n>Experiments on four benchmark datasets with Mistral 7B and Qwen2.5 7B demonstrate that Fast Quiet-STaR consistently outperforms Quiet-STaR in terms of average accuracy.
  arXiv  Detail & Related papers  (2025-05-23T11:14:12Z)
• Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding [11.07450742824775]
  Speculative decoding aims to accelerate the auto-regressive token generation process of a target Large Language Model.<n>Some approaches employ a draft model with multiple heads to predict a sequence of future tokens, where each head handles a token in the sequence.<n>We propose Gumiho, a hybrid model combining serial and parallel heads.
  arXiv  Detail & Related papers  (2025-03-13T07:55:38Z)
• Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Models [2.5464748274973026]
  We present a regression-like loss that operates purely on token level.<n>Our proposed Number Token Loss (NTL) comes in two flavors and minimizes either the $L_p$ norm or the Wasserstein distance.<n>We evaluate the proposed scheme on various mathematical datasets and find that it consistently improves performance in math-related tasks.
  arXiv  Detail & Related papers  (2024-11-04T13:43:24Z)
• FIRP: Faster LLM inference via future intermediate representation prediction [54.897493351694195]
  FIRP generates multiple tokens instead of one at each decoding step. We conduct extensive experiments, showing a speedup ratio of 1.9x-3x in several models and datasets.
  arXiv  Detail & Related papers  (2024-10-27T15:53:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.