Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning

• URL: http://arxiv.org/abs/2604.18639v1
• Date: Sun, 19 Apr 2026 08:02:31 GMT
• Title: Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
• Abstract summary: We introduce a new perspective inspired by cognitive learning theory and propose a novel approach called EasyRL.<n>EasyRL is to simulate the human cognitive acquisition curve by integrating reliable knowledge transfer from easy labeled data.<n> Experimental results on mathematical and scientific benchmarks demonstrate that EasyRL, using only 10% of easy labeled data, consistently outperforms state-of-the-art baselines.
• Score: 59.25637976883812
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Previous LLMs-based RL studies typically follow either supervised learning with high annotation costs, or unsupervised paradigms using voting or entropy-based rewards. However, their performance remains far from satisfactory due to the substantial annotation cost and issues such as model collapse or reward hacking. To address these issues, we introduce a new perspective inspired by cognitive learning theory and propose a novel approach called EasyRL. The core of EasyRL is to simulate the human cognitive acquisition curve by integrating reliable knowledge transfer from easy labeled data with a progressive divide-and-conquer strategy that tackles increasingly difficult unlabeled data. Specifically, we initialize a warm-up model using supervised RL with few-shot labeled data. This is followed by a divide-and-conquer pseudo-labeling strategy on difficult unlabeled data, combining consistency-based selection for low-uncertainty cases and reflection-based resolution for medium-uncertainty cases. Finally, difficulty-progressive self-training with iterative pseudo-labeling and RL further strengthens the model's reasoning capability. EasyRL provides a unified self-evolving framework that facilitates data-efficient post-training of LLMs. Experimental results on mathematical and scientific benchmarks demonstrate that EasyRL, using only 10% of easy labeled data, consistently outperforms state-of-the-art baselines.

Related papers

• Unified Data Selection for LLM Reasoning [66.0118347918729]
  Training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data.<n>We propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5%) highest-entropy tokens in each reasoning sample.
  arXiv  Detail & Related papers  (2026-05-21T12:21:41Z)
• Cog-DRIFT: Exploration on Adaptively Reformulated Instances Enables Learning from Hard Reasoning Problems [70.94268452442147]
  Reinforcement learning from verifiable rewards (RLVR) has improved the reasoning abilities of LLMs.<n>We propose a simple yet effective solution based on task reformulation.<n>We introduce Cog-DRIFT, a framework that constructs reformulated variants and organizes them into an adaptive curriculum.
  arXiv  Detail & Related papers  (2026-04-06T15:38:38Z)
• MiniRec: Data-Efficient Reinforcement Learning for LLM-based Recommendation [50.417769112326546]
  MiniRec is a data selection framework tailored for RL-based large language models (LLMs) recommendation.<n>It evaluates sample learnability using key RL signals -- rewards -- pruning samples that are too easy (too high reward) or too difficult (consistently low reward)
  arXiv  Detail & Related papers  (2026-02-04T07:15:49Z)
• Rewarding the Journey, Not Just the Destination: A Composite Path and Answer Self-Scoring Reward Mechanism for Test-Time Reinforcement Learning [29.778703252962092]
  Reinforcement Learning (RL) has emerged as a powerful paradigm for advancing Large Language Models (LLMs)<n>We develop a novel test-time reward mechanism that operates without external supervision.
  arXiv  Detail & Related papers  (2025-10-20T07:53:51Z)
• SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data [65.56911325914582]
  We propose Self-play Reinforcement Learning (SeRL) to bootstrap Large Language Models (LLMs) training with limited initial data.<n>The proposed SeRL yields results superior to its counterparts and achieves performance on par with those obtained by high-quality data with verifiable rewards.
  arXiv  Detail & Related papers  (2025-05-25T13:28:04Z)
• LEASE: Offline Preference-based Reinforcement Learning with High Sample Efficiency [9.32396532269159]
  This paper proposes a offLine prEference-bAsed RL with high Sample Efficiency (LEASE) algorithm to generate unlabeled preference data.<n>Considering the pretrained reward model may generate incorrect labels for unlabeled data, we design an uncertainty-aware mechanism to ensure the performance of reward model.
  arXiv  Detail & Related papers  (2024-12-30T15:10:57Z)
• Reward Modeling with Weak Supervision for Language Models [12.599789817157188]
  This work introduces weak supervision as a strategy to extend RLHF datasets and enhance reward model performance. By analyzing RLHF datasets to identify imprecise responses, we wrote simple labeling functions and then calibrated a label model to weakly unlabeled data. Our evaluation show that while weak supervision significantly benefits smaller datasets by improving reward model performance, its effectiveness decreases with larger, originally labeled datasets.
  arXiv  Detail & Related papers  (2024-10-28T09:37:58Z)
• Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization [165.98557106089777]
  A key challenge is to enhance the capabilities of large language models (LLMs) amid a looming shortage of high-quality training data. Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets. We then formalize this strategy into a principled framework of Instance-Reweighted Distributionally Robust Optimization.
  arXiv  Detail & Related papers  (2024-02-22T04:10:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.