Dissecting Failure Dynamics in Large Language Model Reasoning

• URL: http://arxiv.org/abs/2604.14528v1
• Date: Thu, 16 Apr 2026 01:49:12 GMT
• Title: Dissecting Failure Dynamics in Large Language Model Reasoning
• Abstract summary: Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation.<n>We find that errors are not uniformly distributed but often originate from a small number of early transition points.<n>We introduce GUARD, a targeted inference-time framework that probes and redirects critical transitions.
• Score: 9.525092839502433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation, yet how their reasoning failures arise remains poorly understood. By analyzing model-generated reasoning trajectories, we find that errors are not uniformly distributed but often originate from a small number of early transition points, after which reasoning remains locally coherent but globally incorrect. These transitions coincide with localized spikes in token-level entropy, and alternative continuations from the same intermediate state can still lead to correct solutions. Based on these observations, we introduce GUARD, a targeted inference-time framework that probes and redirects critical transitions using uncertainty signals. Empirical evaluations across multiple benchmarks confirm that interventions guided by these failure dynamics lead to more reliable reasoning outcomes. Our findings highlight the importance of understanding when and how reasoning first deviates, complementing existing approaches that focus on scaling inference-time computation.

Related papers

• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• Efficient Test-Time Scaling via Temporal Reasoning Aggregation [5.87118425760756]
  We propose TRACE, a training-free framework for efficient test-time scaling.<n>TRACE determines when to terminate reasoning based on temporal aggregation of multi-step evidence.<n>We show that TRACE reduces reasoning token usage by 25-30% on average while maintaining accuracy within 1-2% of full-length reasoning.
  arXiv  Detail & Related papers  (2026-04-19T07:39:40Z)
• Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models [54.16797570104461]
  Diffusion-based language models (dLLMs) have emerged as a promising alternative to autoregressive language models.<n>We investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis.
  arXiv  Detail & Related papers  (2026-04-12T10:26:41Z)
• Early Stopping for Large Reasoning Models via Confidence Dynamics [55.67938134245981]
  Large reasoning models rely on long chain-of-thought generation to solve complex problems.<n>Key challenge is determining when the model should stop reasoning and produce the final answer.<n>We propose CoDE-Stop, an early stopping method that leverages the dynamics of intermediate answer confidence to decide when to terminate reasoning.
  arXiv  Detail & Related papers  (2026-04-06T17:59:45Z)
• Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring [35.58177960646011]
  We propose an early-exit method to mitigate overthinking in LRLMs.<n>Our method delivers the largest performance improvement over vanilla CoT compared to existing early-exit methods.
  arXiv  Detail & Related papers  (2026-03-15T07:00:47Z)
• Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models [66.36240676392502]
  Chain-of-thought (CoT) reasoning has become the standard paradigm for enabling Large Language Models (LLMs) to solve complex problems.<n>Recent studies reveal a sharp performance drop in reasoning hop generalization scenarios.<n>We propose test-time correction of reasoning, a lightweight intervention method that dynamically identifies and deactivates ep heads in the reasoning process.
  arXiv  Detail & Related papers  (2026-01-29T03:24:32Z)
• TRACE: A Framework for Analyzing and Enhancing Stepwise Reasoning in Vision-Language Models [9.607579442309639]
  We introduce TRACE, a framework for Transparent Reasoning And Consistency Evaluation.<n>At its core, TRACEleverages Auxiliary Reasoning Sets to decompose complex problems.<n>Our experiments show that consistency across ARS correlates with final-answer correctness.<n>TRACE defines confidence regions that distinguish reliable from unreliable reasoning paths.
  arXiv  Detail & Related papers  (2025-12-05T18:40:18Z)
• Improving Temporal Understanding Logic Consistency in Video-Language Models via Attention Enhancement [44.654178762186824]
  Large language models (LLMs) often generate self-contradictory outputs.<n>Video-language models (Video-LLMs) fail to provide consistent responses to logically rephrased questions.<n>We propose an attention enhancement method called Temporally Conditioned Attention Sharpening.
  arXiv  Detail & Related papers  (2025-10-09T12:22:06Z)
• Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning [53.25336975467293]
  We present the first theoretical error decomposition analysis of methods such as perplexity and self-consistency.<n>Our analysis reveals a fundamental trade-off: perplexity methods suffer from substantial model error due to the absence of a proper consistency function.<n>We propose Reasoning-Pruning Perplexity Consistency (RPC), which integrates perplexity with self-consistency, and Reasoning Pruning, which eliminates low-probability reasoning paths.
  arXiv  Detail & Related papers  (2025-02-01T18:09:49Z)
• Localisation in quasiperiodic chains: a theory based on convergence of local propagators [68.8204255655161]
  We present a theory of localisation in quasiperiodic chains with nearest-neighbour hoppings, based on the convergence of local propagators. Analysing the convergence of these continued fractions, localisation or its absence can be determined, yielding in turn the critical points and mobility edges. Results are exemplified by analysing the theory for three quasiperiodic models covering a range of behaviour.
  arXiv  Detail & Related papers  (2021-02-18T16:19:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.