RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning

• URL: http://arxiv.org/abs/2604.16694v1
• Date: Fri, 17 Apr 2026 20:51:04 GMT
• Title: RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning
• Abstract summary: Large reasoning models (LRMs) incur substantial inference latency and computational overhead.<n>We propose RankGuide, a framework that improves the efficiency and effectiveness of collaboration through tensor-rank-guided steering.<n>RankGuide reduces latency by up to $1.75times compared to LRM, while maintaining competitive accuracy relative to prior methods.
• Score: 10.68221932401407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large reasoning models (LRMs) enhance problem-solving capabilities by generating explicit multi-step chains of thought (CoT) reasoning; however, they incur substantial inference latency and computational overhead. To mitigate this issue, recent works have explored model collaboration paradigms, where small reasoning models (SRMs) generate intermediate reasoning steps to achieve a better accuracy--latency trade-off. Despite recent progress, effectively and efficiently detecting and mitigating SRM failures in collaborative systems remains a key challenge. To address this issue, we analyze SRM inference in both the generated text and hidden-state spaces, and identify three types of failure modes: \textit{overconfidence}, \textit{uncertainty}, and \textit{heavy revalidation}. Building on these insights, we propose \textbf{RankGuide}, a framework that improves the efficiency and effectiveness of SRM--LRM collaboration through tensor-rank-guided routing and steering. Specifically, RankGuide leverages a routing signal that incorporates tensor-rank signals derived from consecutive hidden states to detect when SRMs are likely to fail and selectively invoke LRMs. In addition, we introduce a tensor-rank-filtered steering vector extraction method to modulate the reasoning trajectory of SRMs, thereby improving their generation quality. By improving both routing and steering through tensor-rank signals, RankGuide enables SRM--LRM collaborative systems to achieve more efficient reasoning with fewer steps and improved accuracy. Experiments on multiple reasoning benchmarks demonstrate the efficacy of RankGuide in reducing latency by up to $1.75\times$ compared to LRM, while maintaining competitive accuracy relative to prior methods.

Related papers

• Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models [11.158010513386666]
  Large reasoning models (LRMs) can solve complex problems through explicit chain-of-thought reasoning processes.<n>LRMs often suffer from overthinking, resulting in redundant token outputs and degraded accuracy.<n>We propose ASAG, which infers the model's reasoning state and adaptively adjusts the generation strategy.
  arXiv  Detail & Related papers  (2026-06-13T02:58:29Z)
• TrigReason: Trigger-Based Collaboration between Small and Large Reasoning Models [59.83398270447905]
  Large Reasoning Models (LRMs) achieve strong performance on complex tasks but suffer from high inference latency due to autoregressive reasoning.<n>Recent work explores using Small Reasoning Models (SRMs) to accelerate LRM inference.<n>TrigReason is a trigger-based collaborative reasoning framework that replaces continuous polling with selective intervention.
  arXiv  Detail & Related papers  (2026-04-16T10:33:00Z)
• Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty [33.04999074297977]
  E-GRM is an efficient generative reward modeling framework grounded in model-internal uncertainty.<n>We show that E-GRM substantially reduces inference cost while consistently improving answer accuracy.
  arXiv  Detail & Related papers  (2026-04-11T07:35:08Z)
• ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training [11.579744186304895]
  Generative reward models (GRMs) have emerged as a promising approach for aligning Large Language Models (LLMs) with human preferences.<n>We propose ConsistRM, a self-training framework that enables effective and stable GRM training without human annotations.
  arXiv  Detail & Related papers  (2026-04-08T18:25:02Z)
• Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation [58.15178451387516]
  Large reasoning models (LRMs) achieved remarkable performance via chain-of-thought (CoT)<n>Recent studies showed that such enhanced reasoning capabilities are at the expense of significantly degraded safety capabilities.<n>We propose a novel safety alignment method that promotes the safety decision-making of LRMs before starting CoT generation.
  arXiv  Detail & Related papers  (2026-03-18T05:21:12Z)
• Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models [39.290072292743226]
  We introduce Mix-GRM, a framework that reconfigures raw rationales into structured B-CoT and D-CoT through a modular synthesis pipeline.<n>Experiments demonstrate that Mix-GRM establishes a new state-of-the-art across five benchmarks, surpassing leading open-source RMs by an average of 8.2%.
  arXiv  Detail & Related papers  (2026-03-02T07:54:29Z)
• Are Reasoning LLMs Robust to Interventions on Their Chain-of-Thought? [79.86483056611105]
  Reasoning LLMs generate step-by-step chains of thought before giving an answer.<n>How robust are these reasoning traces to disruptions that occur within them?<n>We introduce a controlled evaluation framework that perturbs a model's own CoT at fixed timesteps.
  arXiv  Detail & Related papers  (2026-02-07T10:02:58Z)
• From Long to Short: LLMs Excel at Trimming Own Reasoning Chains [48.692414597960244]
  O1/R1 style large reasoning models (LRMs) signal a substantial leap forward over conventional instruction-following LLMs.<n>Recent studies show that LRMs are prone to suffer from overthinking.<n>We propose a test-time scaling method, EDIT, which efficiently guides LRMs to identify the shortest correct reasoning paths at test time.
  arXiv  Detail & Related papers  (2025-09-07T19:00:44Z)
• Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models [50.4652276723694]
  Think-RM generates flexible, self-guided reasoning traces that support advanced capabilities.<n>Think-RM achieves state-of-the-art results on RM-Bench, outperforming both BT RM and vertically scaled GenRM by 8%.
  arXiv  Detail & Related papers  (2025-05-22T05:56:11Z)
• ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning [64.93140713419561]
  Large Reasoning Models (LRMs) perform strongly in complex reasoning tasks via Chain-of-Thought (CoT) prompting, but often suffer from verbose outputs.<n>Existing fine-tuning-based compression methods either operate post-hoc pruning, risking disruption to reasoning coherence, or rely on sampling-based selection.<n>We introduce ConCISE, a framework designed to generate concise reasoning chains, integrating Confidence Injection to boost reasoning confidence, and Early Stopping to terminate reasoning when confidence is sufficient.
  arXiv  Detail & Related papers  (2025-05-08T01:40:40Z)
• Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models [46.75272311751018]
  We propose a novel reward model approach called the Hierarchical Reward Model.<n>It evaluates both individual and consecutive reasoning steps at both fine-grained and coarse-grained levels.<n>It excels at assessing multi-step reasoning coherence, especially when flawed steps are later corrected through self-reflection.
  arXiv  Detail & Related papers  (2025-03-16T15:18:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.