TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation

• URL: http://arxiv.org/abs/2608.00494v1
• Date: Sat, 01 Aug 2026 07:35:06 GMT
• Title: TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation
• Abstract summary: Test-aware Policy Refinement (TaPR) is a framework that transforms execution feedback into a dense per-turn test-pass-ratio reward.<n>Across six models on 219 code-generation problems from LiveCodeBench, TaPR improves the pooled three-turn success rate (Pass@3) by 2.44 percentage points.
• Score: 6.268849049716121
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-turn code agents rely on execution feedback to repair incorrect programs, yet standard reinforcement learning paradigms optimize and evaluate policy performance primarily using single-shot outcome rewards. This misalignment conflates initial code generation with feedback-driven refinement, discards granular execution signals across intermediate turns, and fails to evaluate whether the policy actually acquires self-repair capabilities. We propose Test-aware Policy Refinement (TaPR), a framework that transforms execution feedback into a dense per-turn test-pass-ratio reward under a consistent multi-turn interaction protocol. Across six models on 219 code-generation problems from LiveCodeBench, TaPR improves the pooled three-turn success rate (Pass@3) by 2.44 percentage points. In the predefined 7B/8B high-headroom slice, pooled accuracy increases from 30.25% to 33.56% (+3.31 pp), with 42 improvements and 13 regressions in paired trials. On a matched Qwen3-8B ablation, the dense reward supplies nonzero feedback in all of the first ten steps and reaches a higher Hard-subset peak than outcome-only GRPO within the tested budget, although GRPO nearly matches pooled Pass@3 by step 300. Our primary contribution is a reward-decomposition framework and a turn-aware evaluation protocol that decouple first-shot generation quality from multi-turn repair competence.

Related papers

• Security Tests as Executable Specifications for LLM Code Generation: Benefits, Trade-offs, and Coverage Limits [14.44743150931186]
  We develop a controlled test-feedback scaffold that separates three factors: whether tests are shown upfront, whether failed executions trigger revision, and how failures are selected and represented.<n>Across 2,705 trajectories, 31 task instances, three secure-code benchmarks, 16 CWE categories, and two model families, showing all visible tests upfront increases hidden functional-and-security joint success by 19.3 percentage points on average.<n>In shared-candidate comparisons, structured feedback repairs 80 initially unsuccessful candidates with no joint regressions; fixed raw feedback repairs 83 but causes three regressions.
  arXiv  Detail & Related papers  (2026-08-10T15:37:09Z)
• Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0 [53.71882250666667]
  Most reported gains from agent-optimization methods are one-shot.<n>This does not test the setting that matters for deployed agents.<n>We compare three approaches to agent-harness optimization.
  arXiv  Detail & Related papers  (2026-07-15T16:36:04Z)
• SCOPE: Leveraging Subgoal Critiques for Code Generation [14.869703886733761]
  We present SCOPE, a prover-d subgoal critic for code generation.<n>SCOPE adapts a Lean-oriented prover model to produce three parseable feedback fields for downstream code generation.<n>Our approach combines supervised fine-tuning, process-aligned reinforcement learning (RL), and feedback-guided inference.
  arXiv  Detail & Related papers  (2026-07-07T04:09:41Z)
• When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming [0.0]
  We present an empirical failure-mode study of a compact RLHF pipeline with proximal policy optimization (PPO)<n>We classify matched transitions between checkpoints using the directions of the learned reward, judge scores, and average judge score.<n>A pre-transition logistic model predicts future row-level reward hacking with ROC-AUC 0.821, and row-level analysis finds localized reward hacking that checkpoint averages miss in 3 of 12 settings.
  arXiv  Detail & Related papers  (2026-06-02T06:55:52Z)
• VeriGate: Verifier-Gated Step-Level Supervision for GRPO [51.26100506256885]
  Group Relative Policy Optimization is an effective recipe for training reasoning models with verifier-based outcome rewards.<n>We present VeriGate, a verifier-gated extension of GRPO that addresses these limitations with three design choices.<n>We show that VeriGate improves average accuracy by about 20% and 12% for 1.5B and 7B models respectively.
  arXiv  Detail & Related papers  (2026-05-28T18:20:32Z)
• Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence [56.25095230687242]
  Coding agents often over-trust their own local validation routines and declare success on artifacts that satisfy surface checks.<n>This problem is especially acute in conversion, where prior evaluation is outcome-driven.<n>Across blind conversion attempts, the best system reaches 26.7--28.9% overall pass rate despite Spec pass rates up to 91.1%.<n>This suggests that failures stem more from contract-misaligned self-validation than from limited budget or backbone strength.
  arXiv  Detail & Related papers  (2026-05-27T19:57:15Z)
• A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback [4.062253229926774]
  A-ProS is an autonomous AI agent that solves competitive programming problems through a hybrid multi-model feedback framework.<n>A-ProS achieves over 2x greater gains compared to baseline agent loops.
  arXiv  Detail & Related papers  (2026-05-18T08:55:30Z)
• Step-wise Rubric Rewards for LLM Reasoning [72.17879367869503]
  Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning in large language models.<n>We find 18.2% of steps in correct responses are wrong yet positively rewarded.<n>Across six mathematical reasoning benchmarks, SRaR improves average accuracy over RaR by 3.57 points.
  arXiv  Detail & Related papers  (2026-05-17T07:08:14Z)
• BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models [9.519307176717094]
  Reinforcement learning for program repair is hindered by sparse execution feedback and coarse sequence-level rewards that obscure which edits actually fix bugs.<n>We present BoostAPR, a three-stage framework addressing these challenges.
  arXiv  Detail & Related papers  (2026-05-09T19:31:02Z)
• Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing [79.88256756334327]
  Self-Distillation Policy Optimization (SDPO) addresses this by providing denser, more targeted logit-level supervision.<n>We propose Sample-Routed Policy Optimization (SRPO)<n>SRPO routes correct samples to GRPO's reward-aligned reinforcement and failed samples to SDPO's targeted logit-level correction.
  arXiv  Detail & Related papers  (2026-04-02T17:29:18Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.