Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning

• URL: http://arxiv.org/abs/2510.10494v1
• Date: Sun, 12 Oct 2025 08:03:56 GMT
• Title: Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning
• Abstract summary: We introduce Latent-Trajectory signals that characterize the temporal evolution of a model's internal representations.<n>We show that these signals predict solution accuracy more reliably than both cross-layer metrics and output-based confidence measures.
• Score: 21.58053831545995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reasoning models improve their problem-solving ability through inference-time scaling, allocating more compute via longer token budgets. Identifying which reasoning traces are likely to succeed remains a key opportunity: reliably predicting productive paths can substantially reduce wasted computation and improve overall efficiency. We introduce Latent-Trajectory signals that characterize the temporal evolution of a model's internal representations during the generation of intermediate reasoning tokens. By measuring the overall change in latent representations between the start and end of reasoning, the change accumulated across intermediate steps, and the extent to which these changes advance toward the final state, we show that these signals predict solution accuracy more reliably than both cross-layer metrics and output-based confidence measures. When used to guide answer selection across multiple sampled generations, Latent-Trajectory signals make test-time scaling more effective and efficient than majority voting, reducing token usage by up to 70% while preserving and even improving accuracy by 2.6% on average. Moreover, these predictive signals often emerge early in the reasoning trace, enabling early selection and allocation of compute to the most promising candidates. Our findings contribute not only practical strategies for inference-time efficiency, but also a deeper interpretability perspective on how reasoning processes are represented and differentiated in latent space.

Related papers

• GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning [61.67411833252235]
  We introduce GradCuit, which inserts optimizable latent states at a selected Transformer layer between the hidden representations of the prompt and the generated continuation.<n>Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, GradCuit achieves an average accuracy of 64.5%.<n>For interpretability, token-level gradient attribution reveals that latent influence concentrates on reasoning-connector tokens, while layer analysis identifies early-to-middle Transformer layers as the most effective optimization space.
  arXiv  Detail & Related papers  (2026-08-03T17:55:24Z)
• Efficient Test-Time Scaling via Temporal Reasoning Aggregation [5.87118425760756]
  We propose TRACE, a training-free framework for efficient test-time scaling.<n>TRACE determines when to terminate reasoning based on temporal aggregation of multi-step evidence.<n>We show that TRACE reduces reasoning token usage by 25-30% on average while maintaining accuracy within 1-2% of full-length reasoning.
  arXiv  Detail & Related papers  (2026-04-19T07:39:40Z)
• Early Stopping for Large Reasoning Models via Confidence Dynamics [55.67938134245981]
  Large reasoning models rely on long chain-of-thought generation to solve complex problems.<n>Key challenge is determining when the model should stop reasoning and produce the final answer.<n>We propose CoDE-Stop, an early stopping method that leverages the dynamics of intermediate answer confidence to decide when to terminate reasoning.
  arXiv  Detail & Related papers  (2026-04-06T17:59:45Z)
• Improving reasoning at inference time via uncertainty minimisation [1.9145422469692004]
  We propose a principled strategy that frames reasoning as uncertainty minimisation.<n>Our method selects at each reasoning step, the continuation that maximizes the model's self-certainty.<n>We show that self-certainty maximisation applied to the early steps can explain most of the performance gain.
  arXiv  Detail & Related papers  (2026-03-07T11:39:06Z)
• One-Token Verification for Reasoning Correctness Estimation [31.590898058475464]
  One-Token Verification (OTV) is a computational method that estimates reasoning correctness in a single forward pass during generation.<n>OTV consistently surpasses existing verifiers and reduces token usage by up to $90%$ through correctness-guided early termination.
  arXiv  Detail & Related papers  (2026-03-01T10:09:58Z)
• Towards Anytime-Valid Statistical Watermarking [63.02116925616554]
  We develop the first e-value-based watermarking framework, Anchored E-Watermarking, that unifies optimal sampling with anytime-valid inference.<n>Our framework can significantly enhance sample efficiency, reducing the average token budget required for detection by 13-15% relative to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-19T18:32:26Z)
• Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization [56.59356959631999]
  Gated Perception-Reasoning Optimization (GPRO) is a meta-reasoning controller that dynamically routes computation among three decision paths.<n>GPRO substantially improves both accuracy and efficiency, outperforming recent slow-thinking methods.
  arXiv  Detail & Related papers  (2026-01-07T23:05:17Z)
• Arbitrage: Efficient Reasoning via Advantage-Aware Speculation [71.45710345765528]
  Speculative Decoding accelerates inference by employing a fast but inaccurate draft model to autoregressively propose tokens.<n>But due to unnecessary rejections caused by token mismatches in semantically equivalent steps, traditional token-level Speculative Decoding struggles in reasoning tasks.<n>We propose Arbitrage, a novel step-level speculative generation framework that routes generation dynamically based on the relative advantage between draft and target models.
  arXiv  Detail & Related papers  (2025-12-04T17:50:53Z)
• Efficient Thought Space Exploration through Strategic Intervention [54.35208611253168]
  We propose a novel Hint-Practice Reasoning (HPR) framework that operationalizes this insight through two synergistic components.<n>The framework's core innovation lies in Distributional Inconsistency Reduction (DIR), which dynamically identifies intervention points.<n> Experiments across arithmetic and commonsense reasoning benchmarks demonstrate HPR's state-of-the-art efficiency-accuracy tradeoffs.
  arXiv  Detail & Related papers  (2025-11-13T07:26:01Z)
• Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning [5.37133760455631]
  We introduce a novel entropy-based framework to drive token efficiency in large language models during reasoning tasks.<n>Our approach uses Shannon entropy from token-level logprobs as a confidence signal to enable early stopping.<n>We show that entropy-based confidence calibration represents an emergent property of advanced post-training optimization.
  arXiv  Detail & Related papers  (2025-10-09T12:33:16Z)
• Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search [62.1546099504045]
  We propose a dual-phase test-time scaling framework that separates reasoning into planning and execution.<n>Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately.<n> Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing computation redundant.
  arXiv  Detail & Related papers  (2025-09-29T19:27:23Z)
• On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study [4.319482898846564]
  We study two key factors for improving reasoning in large language models.<n>We train decoder-only transformers on question-trace-answer triples using a custom tokenizer.<n>With the same training-token budget, models trained on inefficient traces generalize better to unseen graphs.
  arXiv  Detail & Related papers  (2025-07-07T18:00:06Z)
• Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models [68.96619605651155]
  Large reasoning models (LRMs) may drastically increase the output length due to overthinking.<n>We propose a dynamic optimization framework that segments model-generated reasoning paths into distinct thinking patterns.<n>Our method achieves up to a 12% accuracy improvement and reducing token usage from approximately 5,000 to 3,000 tokens.
  arXiv  Detail & Related papers  (2025-05-27T20:59:29Z)
• LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling [29.721108461390973]
  We introduce PIR (Perplexity-based Importance Refinement), a principled framework that quantitatively evaluates the importance of each reasoning step.<n>PIR identifies and selectively prunes only low-importance functional steps while preserving progressive reasoning components.<n>Our approach demonstrates strong generalizability across different model sizes, data sources, and token budgets.
  arXiv  Detail & Related papers  (2025-05-25T15:17:57Z)
• Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens [51.90059610606049]
  This paper revisits the efficiency of such reasoning processes through an information-theoretic lens.<n>We propose two metrics, InfoBias and InfoGain, to quantify divergence from ideal reasoning paths and stepwise information contribution.<n>Motivated by these findings, we introduce an entropy-based Adaptive Think strategy that dynamically halts reasoning once confidence is sufficiently high.
  arXiv  Detail & Related papers  (2025-05-23T13:38:56Z)
• Entropy-Gated Branching for Efficient Test-Time Reasoning [21.810952984561116]
  Test-time compute methods can significantly improve the reasoning capabilities and problem-solving accuracy of large language models (LLMs)<n>We propose Entropy-Gated Branching (EGB), which branches only at high-uncertainty steps and prunes expansions with a lightweight verifier.<n>On mathematical and financial reasoning benchmarks, EGB improves accuracy by 22.6% over standard inference while operating 31%-75% faster across math benchmarks than test-time beam search with higher performance.
  arXiv  Detail & Related papers  (2025-03-27T20:18:22Z)
• Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs [3.6696973040141034]
  path-consistency improves inference latency by up to 40.5%, while maintaining task accuracy across various tasks.<n>Our empirical results demonstrate that path-consistency improves inference latency by up to 40.5%, while maintaining task accuracy across various tasks.
  arXiv  Detail & Related papers  (2024-08-25T01:45:53Z)
• Dynamic Transformers Provide a False Sense of Efficiency [75.39702559746533]
  Multi-exit models make a trade-off between efficiency and accuracy, where the saving of computation comes from an early exit. We propose a simple yet effective attacking framework, SAME, which is specially tailored to reduce the efficiency of the multi-exit models. Experiments on the GLUE benchmark show that SAME can effectively diminish the efficiency gain of various multi-exit models by 80% on average.
  arXiv  Detail & Related papers  (2023-05-20T16:41:48Z)
• Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation [87.54604263202941]
  We propose a tiny deep neural network of which partial layers are iteratively exploited for refining its previous estimations. We employ learned gating criteria to decide whether to exit from the weight-sharing loop, allowing per-sample adaptation in our model. Our method consistently outperforms state-of-the-art 2D/3D hand pose estimation approaches in terms of both accuracy and efficiency for widely used benchmarks.
  arXiv  Detail & Related papers  (2021-11-11T23:31:34Z)
• BERT Loses Patience: Fast and Robust Inference with Early Exit [91.26199404912019]
  We propose Patience-based Early Exit as a plug-and-play technique to improve the efficiency and robustness of a pretrained language model. Our approach improves inference efficiency as it allows the model to make a prediction with fewer layers.
  arXiv  Detail & Related papers  (2020-06-07T13:38:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.