Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models

• URL: http://arxiv.org/abs/2603.07777v1
• Date: Sun, 08 Mar 2026 19:40:12 GMT
• Title: Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
• Abstract summary: Modern code generation models exhibit longer outputs, accelerated capability growth, and changed training dynamics.<n>We propose MicroCoder-GRPO, an improved Group Relative Policy Optimization approach with three innovations.<n>MicroCoder-GRPO achieves up to 17.6% relative improvement over strong baselines on LiveCodeBench v6.<n>We release MicroCoder-Dataset, a more challenging training corpus that achieves 3x larger performance gains than mainstream datasets on LiveCodeBench v6 within 300 training steps.
• Score: 104.26904744478884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern code generation models exhibit longer outputs, accelerated capability growth, and changed training dynamics, rendering traditional training methodologies, algorithms, and datasets ineffective for improving their performance. To address these training bottlenecks, we propose MicroCoder-GRPO, an improved Group Relative Policy Optimization approach with three innovations: conditional truncation masking to improve long output potential while maintaining training stability, diversity-determined temperature selection to maintain and encourage output diversity, and removal of KL loss with high clipping ratios to facilitate solution diversity. MicroCoder-GRPO achieves up to 17.6% relative improvement over strong baselines on LiveCodeBench v6, with more pronounced gains under extended context evaluation. Additionally, we release MicroCoder-Dataset, a more challenging training corpus that achieves 3x larger performance gains than mainstream datasets on LiveCodeBench v6 within 300 training steps, and MicroCoder-Evaluator, a robust framework with approximately 25% improved evaluation accuracy and around 40% faster execution. Through comprehensive analysis across more than thirty controlled experiments, we reveal 34 training insights across seven main aspects, demonstrating that properly trained models can achieve competitive performance with larger counterparts.

Related papers

• Layer-adaptive Expert Pruning for Pre-Training of Mixture-of-Experts Large Language Models [13.25114757425841]
  This work introduces a Layer-Adaptive Expert Pruning (LAEP) algorithm for the pre-training stage of Mixture-of-Experts (MoE) Large Language Models (LLMs)<n> Comprehensive experiments demonstrate that LAEP effectively reduces model size and substantially improves pre-training efficiency.<n>In particular, when pre-training the 1010B Base model from scratch, LAEP achieves a 48.3% improvement in training efficiency alongside a 33.3% parameter reduction, while still delivering excellent performance across multiple domains.
  arXiv  Detail & Related papers  (2026-01-20T08:39:04Z)
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward [54.708851958671794]
  We propose a Data-Efficient Policy Optimization pipeline that combines optimized strategies for both offline and online data selection.<n>In offline phase, we curate a high-quality subset of training samples based on diversity, influence, and appropriate difficulty.<n>During online RLVR training, we introduce a sample-level explorability metric to dynamically filter samples with low exploration potential.
  arXiv  Detail & Related papers  (2025-09-01T10:04:20Z)
• Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs [51.21041884010009]
  Ring-lite is a Mixture-of-Experts (MoE)-based large language model optimized via reinforcement learning (RL)<n>Our approach matches the performance of state-of-the-art (SOTA) small-scale reasoning models on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-06-17T17:12:34Z)
• AmorLIP: Efficient Language-Image Pretraining via Amortization [52.533088120633785]
  Contrastive Language-Image Pretraining (CLIP) has demonstrated strong zero-shot performance across diverse downstream text-image tasks.<n>We propose AmorLIP, an efficient CLIP pretraining framework that amortizes expensive computations involved in contrastive learning through lightweight neural networks.
  arXiv  Detail & Related papers  (2025-05-25T05:30:37Z)
• Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization [53.93621974137829]
  Self-Refinement refers to a model's ability to revise its own responses to produce improved outputs.<n>EVOLVE is a framework for eliciting and tracking the evolution of Self-Refinement through iterative training.<n>We demonstrate the potential of leveraging Self-Refinement to achieve broader Self-Improvement of intrinsic model abilities.
  arXiv  Detail & Related papers  (2025-02-08T15:21:55Z)
• Faster and Better 3D Splatting via Group Training [4.7913404251054335]
  Group Training is a strategy that organizes Gaussian primitives into manageable groups, optimizing training efficiency and improving rendering quality.<n>This approach shows universal compatibility with existing 3DGS frameworks, including vanilla 3DGS and Mip-Splatting.<n>Experiments reveal that our straightforward Group Training strategy achieves up to 30% faster convergence and improved rendering quality across diverse scenarios.
  arXiv  Detail & Related papers  (2024-12-10T15:47:17Z)
• GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning [44.401418612374286]
  We introduce a novel soft-pruning method, GDeR, designed to update the training during the process using trainable prototypes. GDeR achieves or surpasses the performance of the full dataset with 30%50% fewer training samples. It also outperforms state-of-the-art pruning methods in imbalanced training and noisy training scenarios.
  arXiv  Detail & Related papers  (2024-10-17T16:56:01Z)
• CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMs [30.441431693349866]
  Existing methods generate vast amounts of synthetic data for fine-tuning, leading to inefficiencies in training. CodeACT introduces the Complexity and Diversity Aware Sampling (CDAS) method to select high-quality training data. CodeACT achieves an 8.6% performance increase on HumanEval, reduces training time by 78%, and decreases peak GPU memory usage by 27%.
  arXiv  Detail & Related papers  (2024-08-05T02:38:48Z)
• Efficient Continual Pre-training by Mitigating the Stability Gap [68.49269649759005]
  We study the behavior of Large Language Models (LLMs) during continual pre-training. We propose three effective strategies to enhance LLM performance within a fixed compute budget. Our strategies improve the average medical task performance of the OpenLlama-3B model from 36.2% to 40.7% with only 40% of the original training budget.
  arXiv  Detail & Related papers  (2024-06-21T02:28:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.