LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

• URL: http://arxiv.org/abs/2608.01804v2
• Date: Tue, 04 Aug 2026 02:24:08 GMT
• Title: LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation
• Abstract summary: Post-training large language models (LLMs) via reinforcement learning (RL) has significantly advanced code generation capabilities.<n>Current state-of-the-art frameworks leverage critic-free paradigms like Group Relative Policy Optimization (GRPO) tied to rule-based verification sandboxes.<n>We introduce LEAPLean Environment-Feedback via Adaptive Pruning, a scalable and computationally efficient multi-turn framework optimized for low-level hardware accelerator alignment.
• Score: 3.4535054007206636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-training large language models (LLMs) via reinforcement learning (RL) has significantly advanced code generation capabilities. To bypass the heavy memory footprint of critic networks, current state-of-the-art frameworks leverage critic-free paradigms like Group Relative Policy Optimization (GRPO) tied to rule-based verification sandboxes. However, applying these frameworks to low-level systems programming, such as CUDA kernel generation-presents severe challenges: binary pass/fail rewards introduce severe signal sparsity, while multi-turn environmental feedback loops suffer from prohibitive compilation latencies and reward dilution across trajectories. In this work, we introduce LEAP (Lean Environment-Feedback via Adaptive Pruning), a scalable and computationally efficient multi-turn RL framework optimized for low-level hardware accelerator alignment. LEAP features Difficulty-Conditioned Pruning (DCP), a dynamic gating mechanism that adaptively cuts off simple and overly catastrophic tasks from multi-turn expansion, focusing resource-heavy compilation and hardware exploration exclusively on high-value, complex tasks. To fully operationalize these paths without manual hyperparameter engineering, we propose a Rank-Based Reward formulation. By deriving scale-free relative advantages from pairwise tournament outcomes within the GRPO rollout group, our method inherently penalizes token inefficiency on simple prompts while maximizing learning gradients on challenging distributions. Empirical evaluations show that LEAP achieves superior first-turn proficiency and robust multi-turn debugging resilience while converging faster than unpruned multi-turn baselines, establishing a practical paradigm for low-level code RL.

Related papers

• Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification [19.490862077232205]
  We propose a data-centric framework to stabilize Reinforcement Learning training.<n>We first introduce Potential-Aware Query Mining (PAQM), which filters data dynamically to focus on the "Distillation Zone"---samples with high potential for capability elicitation.<n>We present Hybrid Stratified Replay (HSR), a novel mechanism that restructures batches by stratifying rollouts based on Path Entropy, a rollout-level confidence proxy, and outcome reward.
  arXiv  Detail & Related papers  (2026-09-07T07:47:00Z)
• V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think [90.69263509098948]
  We introduce Variational GRPO, a method that integrates ELBO-based surrogates with the Group Relative Policy Optimization algorithm.<n>V-GRPO achieves state-of-the-art performance in text-to-image synthesis, while delivering a $2times$ speedup over MixGRPO and a $3times$ speedup over DiffusionNFT.
  arXiv  Detail & Related papers  (2026-04-25T17:03:21Z)
• Cluster-Aware Attention-Based Deep Reinforcement Learning for Pickup and Delivery Problems [6.977990610183958]
  emphCAADRL (Cluster-Aware Attention-based Deep Reinforcement Learning) is a DRL framework that exploits the multi-scale structure of PDP instances.<n>Our method achieves these results with substantially lower inference time than neural collaborative-search baselines.
  arXiv  Detail & Related papers  (2026-03-09T17:31:34Z)
• LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning [38.04066732299875]
  Diffusion Large Language Models (dLLMs) have emerged as a promising paradigm for parallel token generation.<n>Despite their potential, existing dLLMs typically suffer from a rigid accuracy-parallelism trade-off.<n>We propose LightningRL, a post-training framework designed to directly optimize the speed-quality frontier of pre-trained dLLMs.
  arXiv  Detail & Related papers  (2026-03-04T11:43:19Z)
• Parallel Diffusion Solver via Residual Dirichlet Policy Optimization [88.7827307535107]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face significant image quality degradation under a low-dimensional budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as EPD-EPr), a novel ODE solver that mitigates these errors by incorporating multiple gradient parallel evaluations in each step.
  arXiv  Detail & Related papers  (2025-12-28T05:48:55Z)
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective [85.06838178922791]
  Reinforcement Learning (RL) has proven highly effective for autoregressive language models.<n>But adapting these methods to diffusion large language models (dLLMs) presents fundamental challenges.<n>We propose a principled RL framework that treats entire sequence generation as a single action and uses the ELBO as a tractable sequence-level likelihood proxy.
  arXiv  Detail & Related papers  (2025-12-03T13:05:32Z)
• Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shuffle [65.14124923451077]
  Reinforcement learning (RL) has emerged as an effective post-training paradigm for enhancing the reasoning capabilities of multimodal large language model (MLLM)<n>However, current RL pipelines often suffer from training inefficiencies caused by two underexplored issues: Advantage Collapsing and Rollout Silencing.<n>We propose Shuffle-R1, a simple yet principled framework that improves RL fine-tuning efficiency by dynamically restructuring trajectory sampling and batch composition.
  arXiv  Detail & Related papers  (2025-08-07T17:53:47Z)
• REBEL: Reinforcement Learning via Regressing Relative Rewards [59.68420022466047]
  We propose REBEL, a minimalist RL algorithm for the era of generative models.<n>In theory, we prove that fundamental RL algorithms like Natural Policy Gradient can be seen as variants of REBEL.<n>We find that REBEL provides a unified approach to language modeling and image generation with stronger or similar performance as PPO and DPO.
  arXiv  Detail & Related papers  (2024-04-25T17:20:45Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.