Mitigating Overthinking in Large Reasoning Models via Manifold Steering

• URL: http://arxiv.org/abs/2505.22411v1
• Date: Wed, 28 May 2025 14:39:26 GMT
• Title: Mitigating Overthinking in Large Reasoning Models via Manifold Steering
• Abstract summary: Large Reasoning Models (LRMs) exhibit a phenomenon known as overthinking during inference.<n>We propose Manifold Steering, a novel approach that elegantly projects the steering direction onto the low-dimensional activation manifold.<n>Our method reduces output tokens by up to 71% while maintaining and even improving the accuracy on several mathematical benchmarks.
• Score: 32.666911833023526
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in solving complex tasks such as mathematics and coding. However, these models frequently exhibit a phenomenon known as overthinking during inference, characterized by excessive validation loops and redundant deliberation, leading to substantial computational overheads. In this paper, we aim to mitigate overthinking by investigating the underlying mechanisms from the perspective of mechanistic interpretability. We first showcase that the tendency of overthinking can be effectively captured by a single direction in the model's activation space and the issue can be eased by intervening the activations along this direction. However, this efficacy soon reaches a plateau and even deteriorates as the intervention strength increases. We therefore systematically explore the activation space and find that the overthinking phenomenon is actually tied to a low-dimensional manifold, which indicates that the limited effect stems from the noises introduced by the high-dimensional steering direction. Based on this insight, we propose Manifold Steering, a novel approach that elegantly projects the steering direction onto the low-dimensional activation manifold given the theoretical approximation of the interference noise. Extensive experiments on DeepSeek-R1 distilled models validate that our method reduces output tokens by up to 71% while maintaining and even improving the accuracy on several mathematical benchmarks. Our method also exhibits robust cross-domain transferability, delivering consistent token reduction performance in code generation and knowledge-based QA tasks. Code is available at: https://github.com/Aries-iai/Manifold_Steering.

Related papers

• Is Your Model Thinking or Just Stagnating? PUMA: Diagnosing Reasoning Pathology via Phase-Momentum Alignment [13.9489759632504]
  Test-time scaling empowers Large Reasoning Models (LRMs) to tackle complex tasks via extensive Chain-of-Thought (CoT)<n>This paper introduces PUMA, a training-free framework employing a tiered diagnostic architecture.<n>Experiments on LRMs spanning 1.5B to 32B demonstrate that PUMA consistently outperforms state-of-the-art baselines across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-07-19T11:00:59Z)
• When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoning [52.21239821135325]
  We introduce Dynamic Thought Sufficiency in Reasoning (DTSR), a novel framework for efficient reasoning.<n>Inspired by human metacognition, DTSR operates in two stages: Reflection Signal Monitoring, and Thought Sufficiency Check.<n>DTSR reduces reasoning length by 28.9%-34.9% with minimal performance loss, effectively mitigating overthinking.
  arXiv  Detail & Related papers  (2026-04-08T07:56:28Z)
• Internalizing LLM Reasoning via Discovery and Replay of Latent Actions [4.830503861275364]
  Internalization of chain-of-thought processes into hidden states has emerged as a highly efficient paradigm for scaling test-time compute.<n>We propose STIR (Self-Distilled Tools for Internal Reasoning), a framework that reformulates reasoning enhancement as a dynamic latent trajectory control problem.
  arXiv  Detail & Related papers  (2026-02-04T08:44:57Z)
• Guided Verifier: Collaborative Multimodal Reasoning via Dynamic Process Supervision [11.159231524113764]
  Reinforcement Learning (RL) has emerged as a pivotal mechanism for enhancing the complex reasoning capabilities of Multimodal Large Language Models (MLLMs)<n>In this paper, we propose the textbfGuided Verifier framework to address these structural limitations.<n>We develop a specialized data synthesis pipeline targeting multimodal hallucinations, constructing textbfCoRe dataset of process-level negatives and textbfCorrect-guide textbfReasoning trajectories to train the guided verifier.
  arXiv  Detail & Related papers  (2026-02-04T07:38:42Z)
• Towards Understanding Steering Strength [15.203729631608253]
  A popular approach to post-training control of large language models is the steering of intermediate latent representations.<n>In this work, we propose the first theoretical analysis of steering strength.<n>Our analysis reveals surprising behaviors, including non-monotonic effects of steering strength.
  arXiv  Detail & Related papers  (2026-02-02T19:25:37Z)
• Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization [56.59356959631999]
  Gated Perception-Reasoning Optimization (GPRO) is a meta-reasoning controller that dynamically routes computation among three decision paths.<n>GPRO substantially improves both accuracy and efficiency, outperforming recent slow-thinking methods.
  arXiv  Detail & Related papers  (2026-01-07T23:05:17Z)
• Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models [29.56923793047279]
  We introduce Dynamic Outlier Truncation (DOT), a training-time intervention that selectively suppresses redundant tokens.<n>DOT targets only the extreme tail of response lengths within fully correct rollout groups while preserving long-horizon reasoning capabilities.<n>Our method reduces inference token usage by 78% while simultaneously increasing accuracy compared to the initial policy.
  arXiv  Detail & Related papers  (2026-01-07T14:31:07Z)
• Lost at the Beginning of Reasoning [85.17612793300238]
  We show that the first reasoning step exerts a disproportionately large influence on the final prediction.<n>We propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps.
  arXiv  Detail & Related papers  (2025-06-27T09:53:57Z)
• Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancement [101.77467538102924]
  Large reasoning models (LRMs) exhibit overthinking, which hinders efficiency and inflates inference cost.<n>We propose two lightweight methods to enhance LRM efficiency.<n>First, we introduce Efficiency Steering, a training-free activation steering technique that modulates reasoning behavior via a single direction.<n>Second, we develop Self-Rewarded Efficiency RL, a reinforcement learning framework that dynamically balances task accuracy and brevity.
  arXiv  Detail & Related papers  (2025-06-18T17:18:12Z)
• Let LLMs Break Free from Overthinking via Self-Braking Tuning [60.08396797526657]
  Large reasoning models (LRMs) have significantly enhanced their reasoning capabilities by generating longer chains of thought.<n>This performance gain comes at the cost of a substantial increase in redundant reasoning during the generation process.<n>We propose a novel framework, Self-Braking Tuning (SBT), which tackles overthinking from the perspective of allowing the model to regulate its own reasoning process.
  arXiv  Detail & Related papers  (2025-05-20T16:53:40Z)
• SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.190800043449336]
  Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
  arXiv  Detail & Related papers  (2025-04-07T02:42:07Z)
• Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs [77.66717051042032]
  Practitioners have consistently observed three puzzling phenomena in transformer-based large language models. These phenomena are characterized by certain so-called "sink tokens" receiving disproportionately high attention weights. We elucidate the mechanisms behind extreme-token phenomena.
  arXiv  Detail & Related papers  (2024-10-17T17:54:06Z)
• FFHFlow: A Flow-based Variational Approach for Learning Diverse Dexterous Grasps with Shape-Aware Introspection [19.308304984645684]
  We introduce a novel model that can generate diverse grasps for a multi-fingered hand.<n>The proposed idea gains superior performance and higher run-time efficiency against strong baselines.<n>We also demonstrate substantial benefits of greater diversity for grasping objects in clutter and a confined workspace in the real world.
  arXiv  Detail & Related papers  (2024-07-21T13:33:08Z)
• Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL [86.0987896274354]
  We first identify a fundamental pattern, self-excitation, as the primary cause of Q-value estimation divergence in offline RL. We then propose a novel Self-Excite Eigenvalue Measure (SEEM) metric to measure the evolving property of Q-network at training. For the first time, our theory can reliably decide whether the training will diverge at an early stage.
  arXiv  Detail & Related papers  (2023-10-06T17:57:44Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.