SAPO: Self-Adaptive Process Optimization Makes Small Reasoners Stronger

• URL: http://arxiv.org/abs/2601.20312v2
• Date: Mon, 02 Feb 2026 10:36:52 GMT
• Title: SAPO: Self-Adaptive Process Optimization Makes Small Reasoners Stronger
• Abstract summary: Existing self-evolution methods overlook the influence of fine-grained reasoning steps, which leads to the reasoner-verifier gap.<n>Motivated by the Error-Related Negativity (ERN), we propose a Self-Adaptive Process Optimization (SAPO) method for self-improvement in Small Language Models (SLMs)
• Score: 13.044308337439249
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing self-evolution methods overlook the influence of fine-grained reasoning steps, which leads to the reasoner-verifier gap. The computational inefficiency of Monte Carlo (MC) process supervision further exacerbates the difficulty in mitigating the gap. Motivated by the Error-Related Negativity (ERN), which the reasoner can localize error following incorrect decisions, guiding rapid adjustments, we propose a Self-Adaptive Process Optimization (SAPO) method for self-improvement in Small Language Models (SLMs). SAPO adaptively and efficiently introduces process supervision signals by actively minimizing the reasoner-verifier gap rather than relying on inefficient MC estimations. Extensive experiments demonstrate that the proposed method outperforms most existing self-evolution methods on two challenging task types: mathematics and code. Additionally, to further investigate SAPO's impact on verifier performance, this work introduces two new benchmarks for process reward models in both mathematical and coding tasks.

Related papers

• A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement [8.401193963526236]
  We propose a training-free regeneration paradigm that leverages an offline-curated contrastive Reflection Memory (RM)<n>At inference time, the method performs RM-guided self-verification followed by a single RM-guided regeneration, avoiding both iterative correction and multi-sample selection.<n>Experiment results show that our method outperforms prior methods while maintaining low computational cost.
  arXiv  Detail & Related papers  (2026-03-20T19:12:36Z)
• ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models [102.4511331368587]
  ARISE (Adaptive Resolution-aware Scaling Evaluation) is a novel metric designed to assess the test-time scaling effectiveness of large reasoning models.<n>We conduct comprehensive experiments evaluating state-of-the-art reasoning models across diverse domains.
  arXiv  Detail & Related papers  (2025-10-07T15:10:51Z)
• Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancement [101.77467538102924]
  Large reasoning models (LRMs) exhibit overthinking, which hinders efficiency and inflates inference cost.<n>We propose two lightweight methods to enhance LRM efficiency.<n>First, we introduce Efficiency Steering, a training-free activation steering technique that modulates reasoning behavior via a single direction.<n>Second, we develop Self-Rewarded Efficiency RL, a reinforcement learning framework that dynamically balances task accuracy and brevity.
  arXiv  Detail & Related papers  (2025-06-18T17:18:12Z)
• PATS: Process-Level Adaptive Thinking Mode Switching [53.53401063490537]
  Current large-language models (LLMs) typically adopt a fixed reasoning strategy, either simple or complex, for all questions, regardless of their difficulty.<n>This neglect of variation in task and reasoning process complexity leads to an imbalance between performance and efficiency.<n>Existing methods attempt to implement training-free fast-slow thinking system switching to handle problems of varying difficulty, but are limited by coarse-grained solution-level strategy adjustments.<n>We propose a novel reasoning paradigm: Process-Level Adaptive Thinking Mode Switching (PATS), which enables LLMs to dynamically adjust their reasoning strategy based on the difficulty of each step, optimizing the balance between
  arXiv  Detail & Related papers  (2025-05-25T17:58:50Z)
• DeclareAligner: A Leap Towards Efficient Optimal Alignments for Declarative Process Model Conformance Checking [1.4064491732635231]
  This paper introduces DeclareAligner, a novel algorithm that uses the A* search algorithm, an established AI pathfinding technique, to tackle the problem.<n>The proposed method is evaluated using 8,054 synthetic and real-life alignment problems, demonstrating its ability to efficiently compute optimal alignments.
  arXiv  Detail & Related papers  (2025-03-13T15:49:29Z)
• On the Effect of Sampling Diversity in Scaling LLM Inference [57.31028064284527]
  Large language model (LLM) scaling inference is key to unlocking greater performance.<n>Motivated by the observed relationship between solution accuracy and meaningful response diversity, we systematically study the effect of prompt diversity in scaling inference.<n>We show that responses generated from meaningful diverse prompts after Best-of-$N$ selection exhibit significantly lower error rates than those produced from stationary prompts.
  arXiv  Detail & Related papers  (2025-02-16T07:37:58Z)
• Self-Consistent Model-based Adaptation for Visual Reinforcement Learning [27.701421196547674]
  Visual reinforcement learning agents face serious performance declines in real-world applications caused by visual distractions.<n>Existing methods rely on fine-tuning the policy's representations with hand-crafted augmentations.<n>We propose Self-Consistent Model-based Adaptation (SCMA), a novel method that fosters robust adaptation without modifying the policy.
  arXiv  Detail & Related papers  (2025-02-14T05:23:56Z)
• Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling [38.7578639980701]
  Self-improvement methods enable large language models to generate solutions themselves.<n>We find that models tend to over-sample on easy queries and under-sample on queries they have yet to master.<n>We introduce Guided Self-Improvement (GSI), a strategy aimed at improving the efficiency of sampling challenging heavy-tailed data.
  arXiv  Detail & Related papers  (2024-11-01T17:18:45Z)
• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo [55.452453947359736]
  We introduce a novel verification method based on Twisted Sequential Monte Carlo (TSMC)<n>We apply TSMC to Large Language Models by estimating the expected future rewards at partial solutions.<n>This approach results in a more straightforward training target that eliminates the need for step-wise human annotations.
  arXiv  Detail & Related papers  (2024-10-02T18:17:54Z)
• When to Update Your Model: Constrained Model-based Reinforcement Learning [50.74369835934703]
  We propose a novel and general theoretical scheme for a non-decreasing performance guarantee of model-based RL (MBRL) Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns.
  arXiv  Detail & Related papers  (2022-10-15T17:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.