Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance

• URL: http://arxiv.org/abs/2605.15012v1
• Date: Thu, 14 May 2026 16:12:30 GMT
• Title: Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance
• Abstract summary: We propose FEST, a demonstration-guided RLVR algorithm.<n>It attains compelling results with only 128 demonstrations randomly selected from an SFT dataset.<n>We find that three components are vital for the success: supervised signal, on-policy signal, and decaying weights on the few-shot SFT dataset.
• Score: 117.2119290254454
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has achieved great success in developing Large Language Models (LLMs) with chain-of-thought rollouts for many tasks such as math and coding. Nevertheless, RLVR struggles with sample efficiency on difficult problems where correct rollouts are hard to generate. Prior works propose to address this issue via demonstration-guided RLVR, i.e., to conduct Supervised FineTuning (SFT) when RL fails; however, SFT often requires a lot of data, which can be expensive to acquire. In this paper, we propose FEST, a FEw-ShoT demonstration-guided RLVR algorithm. It attains compelling results with only 128 demonstrations randomly selected from an SFT dataset. We find that three components are vital for the success: supervised signal, on-policy signal, and decaying weights on the few-shot SFT dataset to prevent overfitting from multiple-epoch training. On several benchmarks, FEST outperforms baselines with magnitudes less SFT data, even matching their performance with full dataset.

Related papers

• Unified Data Selection for LLM Reasoning [66.0118347918729]
  Training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data.<n>We propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5%) highest-entropy tokens in each reasoning sample.
  arXiv  Detail & Related papers  (2026-05-21T12:21:41Z)
• Decouple before Integration: Test-time Synthesis of SFT and RLVR Task Vectors [26.233592394784868]
  We analyze SFT and RLVR through the lens of task vectors.<n>We propose Decoupled Test-time Synthesis (DoTS) to allow SFT and RLVR checkpoints to be trained independently.
  arXiv  Detail & Related papers  (2026-05-01T12:20:44Z)
• Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models [56.12341509545198]
  Large Language Models (LLMs) are pretrained on massive datasets and later instruction-tuned via supervised fine-tuning (SFT) or reinforcement learning (RL)<n>Best practices emphasize large, diverse pretraining data, whereas post-training operates differently.<n>We theoretically analyze transformers trained on an in-context weight prediction task for linear regression.
  arXiv  Detail & Related papers  (2026-03-01T21:58:09Z)
• Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning [49.22815446849924]
  Large Language Models (LLMs) often struggle with problems that require multi-step reasoning.<n>For small-scale open-source models, Reinforcement Learning with Verifiable Rewards (RLVR) fails when correct solutions are rarely sampled.<n>We propose Supervised Reinforcement Learning (SRL), a framework that reformulates problem solving as generating a sequence of logical "actions"
  arXiv  Detail & Related papers  (2025-10-29T22:05:08Z)
• Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning [49.04912820721943]
  Supervised fine-tuning (SFT) is computationally expensive and sometimes suffers from overfitting or bias amplification.<n>This work studies the online batch selection family that dynamically scores and filters samples during the training process.<n>We develop textbfUDS (Utility-Diversity Sampling), a framework for efficient online batch selection in SFT.
  arXiv  Detail & Related papers  (2025-10-19T15:32:01Z)
• RLSR: Reinforcement Learning with Supervised Reward Outperforms SFT in Instruction Following [4.6740998081727385]
  We propose replacing SFT with RLSR to leverage the extensive SFT dataset in an RL framework.<n>In RLSR, the base model generates multiple responses for each prompt, and reward scores are computed as the cosine similarity in the semantic embedding space between the generated and human-labeled responses.
  arXiv  Detail & Related papers  (2025-10-16T01:13:14Z)
• Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead [20.446287312285648]
  We study whether high SFT scores translate to improved performance after RL.<n>We find high SFT scores can be biased toward simpler or more homogeneous data and are not reliably predictive of subsequent RL gains or scaled-up post-training effectiveness.<n>We study alternative metrics and identify generalization loss on held-out reasoning examples and Pass@large k performance to provide strong proxies for the RL outcome.
  arXiv  Detail & Related papers  (2025-10-02T02:57:00Z)
• Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective [98.45690529036848]
  Post-training algorithms such as Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT) are widely used to adapt multimodal large language models to downstream tasks.<n>While effective at task adaptation, their impact on prior knowledge remains unclear.
  arXiv  Detail & Related papers  (2025-06-30T04:15:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.