DePro: Understanding the Role of LLMs in Debugging Competitive Programming Code

• URL: http://arxiv.org/abs/2603.19399v1
• Date: Thu, 19 Mar 2026 18:46:34 GMT
• Title: DePro: Understanding the Role of LLMs in Debugging Competitive Programming Code
• Abstract summary: DePro is a test-case driven approach that assists programmers by correcting existing code rather than generating new solutions.<n>Experiments on 13 faulty user submissions from Codeforces demonstrate that DePro consistently produces correct solutions.
• Score: 2.2340839344812866
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Debugging consumes a substantial portion of the software development lifecycle, yet the effectiveness of Large Language Models(LLMs) in this task is not well understood. Competitive programming offers a rich benchmark for such evaluation, given its diverse problem domains and strict efficiency requirements. We present an empirical study of LLM-based debugging on competitive programming problems and introduce DePro, a test-case driven approach that assists programmers by correcting existing code rather than generating new solutions. DePro combines brute-force reference generation, stress testing, and iterative LLM-guided refinement to identify and resolve errors efficiently.Experiments on 13 faulty user submissions from Codeforces demonstrate that DePro consistently produces correct solutions, reducing debugging attempts by up to 64% and debugging time by an average of 7.6 minutes per problem compared to human programmers and zero-shot LLM debugging.

Related papers

• DebugTA: An LLM-Based Agent for Simplifying Debugging and Teaching in Programming Education [32.673843958049254]
  In programming and Teaching (DT) task, students receive assistance in correcting their erroneous code.<n>We propose DebugTA, a novel teaching agent with specialized tools for standard code retrieval, variable substitution to align reference code, and an external compiler for real-time code analysis.<n>We show that DebugTA consistently improves teaching effectiveness while significantly reducing computational costs.
  arXiv  Detail & Related papers  (2025-10-13T07:17:18Z)
• RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models [49.83481415540291]
  Large Language Models (LLMs) have exhibited significant proficiency in code debug.<n>This paper introduces Repo Debug, a multi-task and multi-language repository-level code debug dataset.<n>We conduct evaluation experiments on 10 LLMs, where Claude 3.5 Sonnect, the best-performing model, still cannot perform well in repository-level debug.
  arXiv  Detail & Related papers  (2025-09-04T10:13:21Z)
• Do AI models help produce verified bug fixes? [62.985237003585674]
  Large Language Models are used to produce corrections to software bugs.<n>This paper investigates how programmers use Large Language Models to complement their own skills.<n>The results are a first step towards a proper role for AI and LLMs in providing guaranteed-correct fixes to program bugs.
  arXiv  Detail & Related papers  (2025-07-21T17:30:16Z)
• Performance Review on LLM for solving leetcode problems [7.377558533352298]
  This paper presents a comprehensive performance evaluation of Large Language Models (LLMs) in solving programming challenges from Leetcode.<n>We generated solutions with multiple LLMs, including GPT-4 and GPT-3.5-turbo.<n>Our results highlight the strengths and limitations of current LLMs in code generation and problem-solving tasks.
  arXiv  Detail & Related papers  (2025-02-16T08:52:45Z)
• PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback [78.89596149768458]
  Large Language Models (LLMs) are widely adopted for assisting in software development tasks.<n>We propose PerfCodeGen, a training-free framework that enhances the performance of LLM-generated code.
  arXiv  Detail & Related papers  (2024-11-18T06:22:38Z)
• Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation [42.49889252988544]
  Large language model (LLM)-based programming assistants can help improve the productivity of professional software developers, but can also facilitate cheating in introductory computer programming courses. This paper investigates the baseline performance of 5 widely used LLMs on a collection of introductory programming problems, examines adversarial perturbations to degrade their performance, and describes the results of a user study aimed at understanding the efficacy of such perturbations in hindering actual code generation for introductory programming assignments.
  arXiv  Detail & Related papers  (2024-10-12T01:01:00Z)
• Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification [52.095460362197336]
  Large language models (LLMs) struggle with consistent and accurate reasoning. LLMs are trained primarily on correct solutions, reducing their ability to detect and learn from errors. We propose a novel collaborative method integrating Chain-of-Thought (CoT) and Program-of-Thought (PoT) solutions for verification.
  arXiv  Detail & Related papers  (2024-10-05T05:21:48Z)
• What's Wrong with Your Code Generated by Large Language Models? An Extensive Study [92.62952504133926]
  This study evaluated the performance of three leading closed-source LLMs and six popular open-source LLMs on three commonly used benchmarks.<n>We developed a taxonomy of bugs for incorrect codes and analyzed the root cause for common bug types.<n>We propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code.
  arXiv  Detail & Related papers  (2024-07-08T17:27:17Z)
• Leveraging Print Debugging to Improve Code Generation in Large Language Models [63.63160583432348]
  Large language models (LLMs) have made significant progress in code generation tasks. But their performance in tackling programming problems with complex data structures and algorithms remains suboptimal. We propose an in-context learning approach that guides LLMs to debug by using a "print debug" method.
  arXiv  Detail & Related papers  (2024-01-10T18:37:59Z)
• Self-Edit: Fault-Aware Code Editor for Code Generation [46.890689359396724]
  Large language models (LLMs) have demonstrated an impressive ability to generate codes on competitive programming tasks. We propose a generate-and-edit approach named Self-Edit to improve the code quality on the competitive programming task.
  arXiv  Detail & Related papers  (2023-05-06T16:12:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.