An Iterative Test-and-Repair Framework for Competitive Code Generation

• URL: http://arxiv.org/abs/2604.05560v1
• Date: Tue, 07 Apr 2026 08:00:54 GMT
• Title: An Iterative Test-and-Repair Framework for Competitive Code Generation
• Abstract summary: Large language models (LLMs) have made remarkable progress in code generation, but competitive programming remains a challenge.<n>Recent training-based methods have improved code generation by using reinforcement learning (RL) with execution feedback.<n>The more recent framework CURE further incorporates test generation into the training process, jointly training a Coder and a Tester within a single model.
• Score: 9.137158235106943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have made remarkable progress in code generation, but competitive programming remains a challenge. Recent training-based methods have improved code generation by using reinforcement learning (RL) with execution feedback. The more recent framework CURE further incorporates test generation into the training process, jointly training a Coder and a Tester within a single model. At inference time, the Coder generates many candidate programs, and the Tester generates tests from the problem description. The candidate who passes the most of the generated tests is selected as the final answer. However, CURE has two critical limitations. First, the Tester never reads any candidate code, so its tests often fail to expose implementation-specific bugs. Second, the Coder generates every candidate from scratch and never learns to fix a buggy program based on a failed test. To address these limitations, we propose FixAudit, which approaches competitive code generation from a new perspective: starting from a single initial candidate, it iteratively improves the candidate through a targeted test-and-repair debugging cycle. The framework trains one shared model with two specialized roles through four stages: the Fixer, which repairs the current candidate based on a failing test, and the Auditor, which reads the candidate code to generate new tests that expose its remaining bugs. We evaluate FixAudit on three benchmarks: APPS, CodeContests, and xCodeEval. Applied to a 7B model, the framework surpasses the average performance of the larger 32B baseline within the same model family under the zero-shot setting. Compared to strong baselines built on the same 7B base model, FixAudit improves average Pass@1 by 35.1% to 36.8% and average AvgPassRatio by 7.1% to 24.5%.

Related papers

• ExecCritic: Learn to Test, Test to Improve for Coding Agents [68.42713285082912]
  Execution feedback can guide coding agents toward correct repository repairs, but only when the tests capture the behavior requested by the issue.<n>We introduce ExecCritic, combining a test--verify--revise scaffold with a role-specific reinforcement learning recipe for training agents within it.<n>The scaffold separates test construction from source-code repair: a Test agent independently generates repository-native tests, a fail-closed harness qualifies and freezes them, and a Repair agent revises source code from their execution feedback without changing the tests.
  arXiv  Detail & Related papers  (2026-09-08T17:53:37Z)
• Breaking, Stale, or Missing? Benchmarking Coding Agents on Project-Level Test Evolution [20.606877071567958]
  We present TEBench, the first project-level benchmark for test evolution.<n>We construct TEBench through a four-stage pipeline over Defects4J projects.<n>We evaluate seven configurations spanning three industrial agent frameworks.
  arXiv  Detail & Related papers  (2026-05-07T12:31:09Z)
• ZeroCoder: Can LLMs Improve Code Generation Without Ground-Truth Supervision? [13.984583399745157]
  Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful paradigm to improve it through execution-based feedback.<n>Existing work tried to use self-generated tests to ground rewards, but the lack of discriminative tests constrains the effect due to the sub-optimal performance of the model on test generation.<n>We present ZeroCoder, a fully label-free co-evolutionary framework that jointly trains a Coder and a Tester using execution feedback from self-generated code-test interactions.
  arXiv  Detail & Related papers  (2026-04-09T06:24:54Z)
• Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learning [54.95476453942411]
  Reinforcement learning for code generation relies on verifiable rewards from unit test pass rates.<n>Recent self-play methods unify code and test generation in a single model.<n>Code-A1 achieves code generation performance matching or exceeding models trained on human-annotated tests.
  arXiv  Detail & Related papers  (2026-03-16T17:58:13Z)
• AutoCode: LLMs as Problem Setters for Competitive Programming [94.71566758494787]
  We introduce AutoCode, which uses multiple rounds of validation to yield competition-grade problem statements and test cases.<n>On held-out problems, AutoCode test suites approach 99% consistency with official judgments.
  arXiv  Detail & Related papers  (2025-09-29T17:59:03Z)
• Alignment with Fill-In-the-Middle for Enhancing Code Generation [56.791415642365415]
  We propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases.<n>Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench.
  arXiv  Detail & Related papers  (2025-08-27T03:15:53Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks.<n>However, improvement is plateauing due to the exhaustion of readily available high-quality data.<n>We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• AugmenTest: Enhancing Tests with LLM-Driven Oracles [2.159639193866661]
  AugmenTest is an approach leveraging Large Language Models to infer correct test oracles based on available documentation of the software under test.<n>AugmenTest includes four variants: Simple Prompt, Extended Prompt, RAG with a generic prompt (without the context of class or method under test), and RAG with Simple Prompt, each offering different levels of contextual information to the LLMs.<n>Results show that in the most conservative scenario, AugmenTest's Extended Prompt consistently outperformed the Simple Prompt, achieving a success rate of 30% for generating correct assertions.
  arXiv  Detail & Related papers  (2025-01-29T07:45:41Z)
• ConAIR:Consistency-Augmented Iterative Interaction Framework to Enhance the Reliability of Code Generation [17.68163468068264]
  We propose a Consistency-Augmented Iterative Interaction Framework to enhance the reliability of Code Generation, ConAIR. We show that with minimal human effort, performance can be significantly enhanced.
  arXiv  Detail & Related papers  (2024-11-23T15:26:24Z)
• TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark [24.14654309612826]
  TestGenEval comprises 68,647 tests from 1,210 code and test file pairs across 11 well-maintained Python repositories.<n>It covers initial tests authoring, test suite completion, and code coverage improvements.<n>We evaluate several popular models, with sizes ranging from 7B to 405B parameters.
  arXiv  Detail & Related papers  (2024-10-01T14:47:05Z)
• TestART: Improving LLM-based Unit Testing via Co-evolution of Automated Generation and Repair Iteration [7.509927117191286]
  Large language models (LLMs) have demonstrated remarkable capabilities in generating unit test cases.<n>We propose TestART, a novel unit test generation method.<n>TestART improves LLM-based unit testing via co-evolution of automated generation and repair iteration.
  arXiv  Detail & Related papers  (2024-08-06T10:52:41Z)
• CodeT: Code Generation with Generated Tests [49.622590050797236]
  We explore the use of pre-trained language models to automatically generate test cases. CodeT executes the code solutions using the generated test cases, and then chooses the best solution. We evaluate CodeT on five different pre-trained models with both HumanEval and MBPP benchmarks.
  arXiv  Detail & Related papers  (2022-07-21T10:18:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.