Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models

• URL: http://arxiv.org/abs/2507.15512v1
• Date: Mon, 21 Jul 2025 11:28:09 GMT
• Title: Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models
• Authors: Kaiyan Chang, Yonghao Shi, Chenglong Wang, Hang Zhou, Chi Hu, Xiaoqian Liu, Yingfeng Luo, Yuan Ge, Tong Xiao, Jingbo Zhu,
• Abstract summary: Test-Time Scaling is a promising approach to progressively elicit the model's intelligence during inference.<n>In this paper, we focus on training-free TTS methods for reasoning.<n>We introduce a novel inference paradigm called Hybrid Test-Time Scaling.
• Score: 53.567732520169
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test-Time Scaling (TTS) is a promising approach to progressively elicit the model's intelligence during inference. Recently, training-based TTS methods, such as continued reinforcement learning (RL), have further surged in popularity, while training-free TTS methods are gradually fading from prominence. However, the additional computation overhead of training amplifies the burden on test-time scaling. In this paper, we focus on training-free TTS methods for reasoning. We first design Conditional Step-level Self-refinement, a fine-grained sequential scaling method guided by process verification. On top of its effectiveness, we further combine it with other classical parallel scaling methods at the step level, to introduce a novel inference paradigm called Hybrid Test-Time Scaling. Extensive experiments on five instruction-tuned LLMs across different scales (3B-14B) and families demonstrate that hybrid strategy incorporating various training-free TTS methods at a fine granularity has considerable potential for expanding the reasoning performance boundaries of LLMs.

Related papers

• Beyond First-Order: Training LLMs with Stochastic Conjugate Subgradients and AdamW [2.028622227373579]
  gradient-based descent (SGD) have long been central to training large language models (LLMs)<n>This paper proposes a conjugate subgradient method together with adaptive sampling specifically for training LLMs.
  arXiv  Detail & Related papers  (2025-07-01T23:30:15Z)
• Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space [82.75174050101108]
  We introduce LatentSeek, a framework that enhances reasoning through Test-Time Instance-level Adaptation (TTIA) within the model's latent space.<n>LatentSeek is evaluated on a range of reasoning benchmarks, including GSM8K, MATH-500, and AIME2024.<n>Results show that LatentSeek consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2025-05-19T16:26:02Z)
• LESA: Learnable LLM Layer Scaling-Up [57.0510934286449]
  Training Large Language Models (LLMs) from scratch requires immense computational resources, making it prohibitively expensive.<n>Model scaling-up offers a promising solution by leveraging the parameters of smaller models to create larger ones.<n>We propose textbfLESA, a novel learnable method for depth scaling-up.
  arXiv  Detail & Related papers  (2025-02-19T14:58:48Z)
• VFScale: Intrinsic Reasoning through Verifier-Free Test-time Scalable Diffusion Model [7.250494262573953]
  We introduce the Verifier-free Test-time scalable Diffusion Model (VFScale) to achieve scalable intrinsic reasoning.<n>On challenging reasoning tasks of Maze and Sudoku, we demonstrate the effectiveness of VFScale's training objective and scalable inference method.<n>In particular, trained with Maze sizes of up to $6times6$, our VFScale solves 88% of Maze problems with much larger sizes of $15times15$, while standard diffusion model completely fails.
  arXiv  Detail & Related papers  (2025-02-04T04:07:48Z)
• SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling [44.11609084435251]
  This paper introduces Self-Enhanced Test-Time Scaling (SETS), a new approach that overcomes limitations by strategically combining parallel and sequential techniques.<n>SETS exploits the inherent self-verification and self-correction capabilities of Large Language Models, unifying sampling, verification, and correction within a single framework.
  arXiv  Detail & Related papers  (2025-01-31T17:03:16Z)
• SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning [73.93639228235622]
  Continual Learning with foundation models has emerged as a promising paradigm to exploit abundant knowledge acquired during pre-training for tackling sequential tasks.<n>Existing prompt-based and Low-Rank Adaptation-based (LoRA-based) methods often require expanding a prompt/LoRA pool or retaining samples of previous tasks.<n>We propose Scalable Decoupled LoRA (SD-LoRA) for class incremental learning, which continually separates the learning of the magnitude and direction of LoRA components without rehearsal.
  arXiv  Detail & Related papers  (2025-01-22T20:00:41Z)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling [52.34735382627312]
  Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks.<n>Existing approaches mainly rely on imitation learning and struggle to achieve effective test-time scaling.<n>We present T1 to scale reinforcement learning by encouraging exploration and understand inference scaling.
  arXiv  Detail & Related papers  (2025-01-20T18:33:33Z)
• SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training [68.7896349660824]
  We present an in-depth analysis of the progressive overfitting problem from the lens of Seq FT. Considering that the overly fast representation learning and the biased classification layer constitute this particular problem, we introduce the advanced Slow Learner with Alignment (S++) framework. Our approach involves a Slow Learner to selectively reduce the learning rate of backbone parameters, and a Alignment to align the disjoint classification layers in a post-hoc fashion.
  arXiv  Detail & Related papers  (2024-08-15T17:50:07Z)
• Test-Time Training on Graphs with Large Language Models (LLMs) [68.375487369596]
  Test-Time Training (TTT) has been proposed as a promising approach to train Graph Neural Networks (GNNs) Inspired by the great annotation ability of Large Language Models (LLMs) on Text-Attributed Graphs (TAGs), we propose to enhance the test-time training on graphs with LLMs as annotators. A two-stage training strategy is designed to tailor the test-time model with the limited and noisy labels.
  arXiv  Detail & Related papers  (2024-04-21T08:20:02Z)
• Test-Time Adaptation with Perturbation Consistency Learning [32.58879780726279]
  We propose a simple test-time adaptation method to promote the model to make stable predictions for samples with distribution shifts. Our method can achieve higher or comparable performance with less inference time over strong PLM backbones.
  arXiv  Detail & Related papers  (2023-04-25T12:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.