SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling

• URL: http://arxiv.org/abs/2501.19306v2
• Date: Mon, 03 Feb 2025 06:21:08 GMT
• Title: SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling
• Authors: Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, Sercan Ö Arık,
• Abstract summary: Self-Enhanced Test-Time Scaling (SETS) is a novel method that integrates sampling, self-verification, and self-correction into a unified framework.<n>SETS achieves significant performance improvements and more favorable test-time scaling laws.
• Score: 36.76945841119825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in Large Language Models (LLMs) have created new opportunities to enhance performance on complex reasoning tasks by leveraging test-time computation. However, conventional approaches such as repeated sampling with majority voting or reward model scoring, often face diminishing returns as test-time compute scales, in addition to requiring costly task-specific reward model training. In this paper, we present Self-Enhanced Test-Time Scaling (SETS), a novel method that leverages the self-verification and self-correction capabilities of recent advanced LLMs to overcome these limitations. SETS integrates sampling, self-verification, and self-correction into a unified framework, enabling efficient and scalable test-time computation for improved capabilities at complex tasks. Through extensive experiments on challenging planning and reasoning benchmarks, compared to the alternatives, we demonstrate that SETS achieves significant performance improvements and more favorable test-time scaling laws.

Related papers

• T1: Tool-integrated Self-verification for Test-time Compute Scaling in Small Language Models [9.674458633565111]
  We investigate whether small language models (sLMs) can reliably self-verify their outputs under test-time scaling. We propose Tool-integrated self-verification (T1), which delegates-heavy verification steps to external tools, such as a code interpreter. Our theoretical analysis shows that tool integration reduces memorization demands and improves test-time scaling performance.
  arXiv  Detail & Related papers  (2025-04-07T04:01:17Z)
• Adaptive Rectification Sampling for Test-Time Compute Scaling [5.085583751997239]
  We propose Adaptive Rectification Sampling (AR-Sampling) to guide large language models to self-correction. Our approach enables the models to rethink in more fine-grained level, improving the accuracy of solutions.
  arXiv  Detail & Related papers  (2025-04-02T02:57:52Z)
• Scalable Best-of-N Selection for Large Language Models via Self-Certainty [65.31658824274894]
  Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models. We propose self-certainty, a novel and efficient metric to estimate response quality without requiring external reward models. Our findings establish self-certainty as a practical and efficient way for improving LLM reasoning capabilities.
  arXiv  Detail & Related papers  (2025-02-25T19:08:07Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks. However, improvement is plateauing due to the exhaustion of readily available high-quality data. We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• Iterative Deepening Sampling for Large Language Models [27.807695570974644]
  Training models to achieve effective self-correction and self-correction remains a significant challenge. We propose a novel iterative sampling algorithm framework designed to enhance self-correction and generate higher-quality samples.
  arXiv  Detail & Related papers  (2025-02-08T04:39:51Z)
• S-LoRA: Scalable Low-Rank Adaptation for Class Incremental Learning [73.93639228235622]
  Continual Learning with foundation models has emerged as a promising approach to harnessing the power of pre-trained models for sequential tasks.<n>We propose a Scalable Low-Rank Adaptation (S-LoRA) method for CL (in particular class incremental learning), which incrementally decouples the learning of the direction and magnitude of LoRA parameters.<n>Our theoretical and empirical analysis demonstrates that S-LoRA tends to follow a low-loss trajectory that converges to an overlapped low-loss region, resulting in an excellent stability-plasticity trade-off in CL.
  arXiv  Detail & Related papers  (2025-01-22T20:00:41Z)
• Self-Improvement in Language Models: The Sharpening Mechanism [70.9248553790022]
  We offer a new perspective on the capabilities of self-improvement through a lens we refer to as sharpening. Motivated by the observation that language models are often better at verifying response quality than they are at generating correct responses, we formalize self-improvement as using the model itself as a verifier during post-training. We analyze two natural families of self-improvement algorithms based on SFT and RLHF.
  arXiv  Detail & Related papers  (2024-12-02T20:24:17Z)
• Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo [55.452453947359736]
  We introduce a novel verification method based on Twisted Sequential Monte Carlo (TSMC) We apply TSMC to Large Language Models by estimating the expected future rewards at partial solutions. This approach results in a more straightforward training target that eliminates the need for step-wise human annotations.
  arXiv  Detail & Related papers  (2024-10-02T18:17:54Z)
• Active Testing of Large Language Model via Multi-Stage Sampling [17.89896012553348]
  AcTracer is an active testing framework tailored for large language models (LLMs) It strategically selects a small subset of test data to achieve a nearly optimal performance estimation. Our experiment results demonstrate that AcTracer achieves state-of-the-art performance compared to existing methods.
  arXiv  Detail & Related papers  (2024-08-07T06:17:48Z)
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters [27.656263126925815]
  We study the scaling of inference-time computation in LLMs. We find that in both cases, the effectiveness of different approaches to scaling test-time compute critically varies depending on the difficulty of the prompt.
  arXiv  Detail & Related papers  (2024-08-06T17:35:05Z)
• Efficient Test-Time Model Adaptation without Forgetting [60.36499845014649]
  Test-time adaptation seeks to tackle potential distribution shifts between training and testing data. We propose an active sample selection criterion to identify reliable and non-redundant samples. We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
  arXiv  Detail & Related papers  (2022-04-06T06:39:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.