DE-Venus: A Data-Efficient RLVR Framework for Large Language Models

• URL: http://arxiv.org/abs/2609.03324v1
• Date: Thu, 03 Sep 2026 03:23:24 GMT
• Title: DE-Venus: A Data-Efficient RLVR Framework for Large Language Models
• Abstract summary: Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost of obtaining reliable targets at scale.<n>We present DE-Venus, a unified framework for data-efficient RLVR that treats supervision as evolving state across data preparation and policy optimization.
• Score: 58.53135628199626
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost of obtaining reliable targets at scale. Existing methods address sample selection, incomplete supervision, or noisy labels separately, often entangling supervision logic with distributed training and hindering controlled comparison and reuse. We present DE-Venus, a unified framework for data-efficient RLVR that treats supervision as evolving state across data preparation and policy optimization. It organizes this lifecycle into three modules: Active Data Selection allocates training and annotation budgets; Weak Supervision Construction derives learning signals from unlabeled examples; and Training-Time Supervision Refinement filters or corrects unreliable supervision. DE-Venus supports seven representative methods and a data-selection pipeline by expressing method-specific decisions as offline dataset transitions or online transformations of targets, rewards, batches, and advantages while preserving verl's distributed execution contracts. Across public benchmarks and three business scenarios, separate configurations preserve or improve model quality with only 10% of labels or as little as 13% of relevant data; selected business configurations also reduce observed convergence steps by 63%--75%. DE-Venus thus reduces annotation and training costs without sacrificing scalable RL execution.

Related papers

• Selectivity Drives Efficiency: Dataset Pruning for Visual Place Recognition [52.52131290631774]
  We propose a place-wise dataset pruning framework tailored for visual place recognition (VPR)<n>Our method treats each place as the basic pruning unit and introduces two novel metrics, i.e., intra-place diversity (IPD) and inter-place similarity (IPS)<n>Experiments demonstrate that our method consistently outperforms state-of-the-art DP baselines under different pruning ratios.
  arXiv  Detail & Related papers  (2026-07-16T12:16:00Z)
• Optimizing Visual Generative Models via Distribution-wise Rewards [57.46109819595665]
  We present a novel framework that finetunes generative models using distribution-wise rewards, ensuring better alignment with real-world data distributions.<n>Our approach significantly improves FID-50K across various base models, from 5.77 for SiT and from 3.74 to 3.52 for EDM2.
  arXiv  Detail & Related papers  (2026-07-02T15:08:56Z)
• GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling [29.20321049470364]
  Reinforcement learning with verifiable rewards (RLVR) significantly advances LLM reasoning.<n>It faces a dilemma: standard supervised scaling is throttled by high annotation costs, while unsupervised alternatives suffer from severe model collapse.<n>Recent semi-supervised RLVR methods address this by using a small labeled set to guide unlabeled data, achieving a promising trade-off between training efficacy and annotation cost.<n>We propose GeoMin, which models global feature on distributions labeled data to decode the structural discrepancy between correct and incorrect rollouts.
  arXiv  Detail & Related papers  (2026-06-03T06:47:50Z)
• Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots [29.141492387722963]
  "Pick in the dark" setup for RLVR aims to select, without prior supervision, unlabeled samples that are most beneficial for training and worthy of annotation.<n>We propose PivotTrace, a three-way data triage framework that leverages attention dynamics to trace metacognitive pivots during reasoning.
  arXiv  Detail & Related papers  (2026-06-03T06:34:42Z)
• When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewards [49.25249414962884]
  We propose Reinforcement Learning with Active Verifiable Rewards (RLAVR)<n>RLAVR actively acquires ground-truth labels for a small set of selected samples and integrates them with pseudo-labels.<n>Building on this, we introduce Correction-Aware Reliability Estimation for RLAVR (CARE)
  arXiv  Detail & Related papers  (2026-05-25T13:55:12Z)
• How Faithful Is Trajectory-Based Data Attribution? Error Sources, Remedies, and Practical Guidelines [21.57090069950487]
  Trajectory-based data attribution methods estimate the influence of training samples on model predictions by unrolling the training trajectory.<n>There is a lack of comprehensive error analysis of these methods, raising concerns about method faithfulness and hindering reliable deployment.<n>We provide the first systematic analysis of error sources in trajectory-based data attribution, together with concrete remedies to mitigate them and practical guidelines for downstream use.
  arXiv  Detail & Related papers  (2026-05-12T09:50:45Z)
• Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Method [10.014138113199616]
  Data selection is one of the fundamental problems in neural network training.<n>Data Value Contribution is a novel budget-aware method for selecting data for training.<n>Our method consistently outperforms existing approaches under various budget constraints.
  arXiv  Detail & Related papers  (2025-10-24T09:33:19Z)
• RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment [10.284993431741377]
  We introduce the concept of epsilon-sample cover, which quantifies sample redundancy based on inter-sample relationships.<n>We reformulate data selection as a reinforcement learning process and propose RL-Selector.<n>Our method consistently outperforms existing state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-06-26T06:28:56Z)
• Data-Driven Offline Decision-Making via Invariant Representation Learning [97.49309949598505]
  offline data-driven decision-making involves synthesizing optimized decisions with no active interaction. A key challenge is distributional shift: when we optimize with respect to the input into a model trained from offline data, it is easy to produce an out-of-distribution (OOD) input that appears erroneously good. In this paper, we formulate offline data-driven decision-making as domain adaptation, where the goal is to make accurate predictions for the value of optimized decisions.
  arXiv  Detail & Related papers  (2022-11-21T11:01:37Z)
• VFed-SSD: Towards Practical Vertical Federated Advertising [53.08038962443853]
  We propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data. Our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.
  arXiv  Detail & Related papers  (2022-05-31T17:45:30Z)
• Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding [62.17020485045456]
  It is commonly assumed in semi-supervised learning (SSL) that the unlabeled data are drawn from the same distribution as that of the labeled ones. We propose to selectively utilize unlabeled data through sample weighting, so that only conducive unlabeled data would be prioritized.
  arXiv  Detail & Related papers  (2022-05-02T16:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.