Learning to Refine Hidden States for Reliable LLM Reasoning

• URL: http://arxiv.org/abs/2606.17524v1
• Date: Tue, 16 Jun 2026 05:03:27 GMT
• Title: Learning to Refine Hidden States for Reliable LLM Reasoning
• Abstract summary: Large language models show strong reasoning ability, but their internal reasoning process can remain unstable in complex multi-step settings.<n>We propose ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations before decoding.
• Score: 0.21485350418225244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models show strong reasoning ability, but their internal reasoning process can remain unstable in complex multi-step settings, where early hidden-state errors may propagate to incorrect predictions. We propose ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations before decoding. ReLAR maintains a compact latent reasoning state and uses learned depth and action controllers to adaptively determine both the number and direction of refinement steps. The controllers are trained with a policy gradient objective based on step-wise likelihood improvement, enabling efficient input-dependent reasoning without explicit chain-of-thought generation. Experiments on medical, mathematical, multi-hop reasoning, and open-ended generation benchmarks show that ReLAR improves accuracy, generation quality, and reasoning stability with substantially lower inference overhead than explicit reasoning baselines.

Related papers

• CARE: Competence-Aware Reward Shaping for Adaptive Reasoning Length in Video-MLLMs [50.189987475377656]
  We propose CARE, a competence-aware reward shaping framework for adaptive reasoning length optimization in multimodal reasoning.<n> CARE maintains a smoothed competence estimate via an exponential moving average of pass rates, and uses it to route training into progressive stages.<n>Experiments on multiple video reasoning and general video understanding benchmarks demonstrate that CARE consistently improves reasoning accuracy, stabilizes reinforcement learning, and significantly enhances token efficiency.
  arXiv  Detail & Related papers  (2026-06-18T08:28:26Z)
• STRIDE: Learnable Stepwise Language Feedback for LLM Reasoning [80.78140312980484]
  We propose language-driven stepwise trajectory redirection, termed as STRIDE.<n>We co-train a generator and a generative verifier using only outcome-based rewards, eliminating external annotations.<n> Experiments on diverse reasoning benchmarks show that STRIDE significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-05-13T11:04:31Z)
• SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoning [12.51459553492698]
  Self-Verification and Self-Rectification (SVSR) is a unified framework that integrates self-verification and self-rectification into the model's reasoning pipeline.<n>SVSR substantially improves robustness and reliability in complex visual understanding and multimodal reasoning tasks.
  arXiv  Detail & Related papers  (2026-04-11T14:25:17Z)
• Learning Structured Reasoning via Tractable Trajectory Control [99.75278337895024]
  Ctrl-R is a framework for learning structured reasoning via tractable trajectory control.<n>We show that Ctrl-R enables effective exploration and internalization of previously unattainable reasoning patterns.
  arXiv  Detail & Related papers  (2026-03-02T09:18:19Z)
• Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
  Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
  arXiv  Detail & Related papers  (2026-02-13T02:13:45Z)
• Structured Reasoning for Large Language Models [59.215789462977206]
  We propose Structured Reasoning (SCR), a framework that decouples reasoning trajectories into explicit, evaluable, and trainable components.<n>SCR substantially improves reasoning efficiency and self-verification.<n>Compared with existing reasoning paradigms, it reduces output token length by up to 50%.
  arXiv  Detail & Related papers  (2026-01-12T04:04:01Z)
• Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization [56.59356959631999]
  Gated Perception-Reasoning Optimization (GPRO) is a meta-reasoning controller that dynamically routes computation among three decision paths.<n>GPRO substantially improves both accuracy and efficiency, outperforming recent slow-thinking methods.
  arXiv  Detail & Related papers  (2026-01-07T23:05:17Z)
• Reflective Confidence: Correcting Reasoning Flaws via Online Self-Correction [14.164508061248775]
  Large language models (LLMs) have achieved strong performance on complex reasoning tasks using techniques such as chain-of-thought and self-consistency.<n>We propose reflective confidence, a novel reasoning framework that transforms low-confidence signals from termination indicators into reflection triggers.<n> Experiments on mathematical reasoning benchmarks, including AIME 2025, demonstrate significant accuracy improvements over advanced early-stopping baselines at comparable computational cost.
  arXiv  Detail & Related papers  (2025-12-21T05:35:07Z)
• Stabilizing Reinforcement Learning for Honesty Alignment in Language Models on Deductive Reasoning [27.42733470720954]
  We propose a reinforcement learning method that injects ground truth trajectories into rollouts, preventing early training collapse.<n>Our results demonstrate that this method stabilizes learning and significantly improves the overall reasoning performance.
  arXiv  Detail & Related papers  (2025-11-12T11:34:19Z)
• Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization [5.674809920704963]
  Latent Thought Policy Optimization enhances LLM reasoning entirely at test time.<n>Experiments show that LTPO not only matches or surpasses strong baselines on standard tasks but also demonstrates remarkable robustness where others fail.<n>Most notably, on highly challenging AIME benchmarks where existing latent reasoning baselines collapse to near-zero accuracy, LTPO delivers substantial improvements.
  arXiv  Detail & Related papers  (2025-10-05T12:50:39Z)
• Certainty-Guided Reasoning in Large Language Models: A Dynamic Thinking Budget Approach [0.15749416770494704]
  We show that Certainty-Guided Reasoning (CGR) improves baseline accuracy while reducing token usage.<n>CGR can eliminate millions of tokens in aggregate, with tunable trade-offs between certainty thresholds and efficiency.<n>By integrating confidence into the reasoning process, CGR makes large reasoning language models more adaptive, trustworthy, and resource efficient.
  arXiv  Detail & Related papers  (2025-09-09T14:57:15Z)
• Lost at the Beginning of Reasoning [85.17612793300238]
  We show that the first reasoning step exerts a disproportionately large influence on the final prediction.<n>We propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps.
  arXiv  Detail & Related papers  (2025-06-27T09:53:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.