Automated Program Repair of Uncompilable Student Code

• URL: http://arxiv.org/abs/2510.06187v1
• Date: Tue, 07 Oct 2025 17:46:33 GMT
• Title: Automated Program Repair of Uncompilable Student Code
• Abstract summary: This study investigates automated program repair as a strategy to recover uncompilable code.<n>We assess large language models (LLMs) as repair agents, including GPT-5 (OpenAI), Claude 3.5 Haiku (Anthropic) and Gemini 2.5 Flash (Google)<n>We find that while all three LLMs are capable of producing compilable repairs, their behavior diverges in how well they preserve students' control flow and code structure.
• Score: 0.6039261200107887
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A significant portion of student programming submissions in CS1 learning environments are uncompilable, limiting their use in student modeling and downstream knowledge tracing. Traditional modeling pipelines often exclude these cases, discarding observations of student learning. This study investigates automated program repair as a strategy to recover uncompilable code while preserving students' structural intent for use in student modeling. Within this framework, we assess large language models (LLMs) as repair agents, including GPT-5 (OpenAI), Claude 3.5 Haiku (Anthropic), and Gemini 2.5 Flash (Google), under high- and low-context prompting conditions. Repairs were evaluated for compilability, edit distance, and preservation of students' original structure and logic. We find that while all three LLMs are capable of producing compilable repairs, their behavior diverges in how well they preserve students' control flow and code structure, which affects their pedagogical utility. By recovering uncompilable submissions, this work enables richer and more comprehensive analyses of learners' coding processes and development over time.

Related papers

• Beyond AI Delegation: A Prompt Pattern Framework for Productive Struggle and Evaluative Judgement in Secure Coding Education [0.45706494756895794]
  Large language models make it easy for students to delegate writing, analysis, and problem-solving to automated systems.<n>We introduce a practical framework that helps educators keep GenAI in the course without removing the cognitive demands that make it worthwhile.
  arXiv  Detail & Related papers  (2026-05-23T07:35:49Z)
• Understanding by Reconstruction: Reversing the Software Development Process for LLM Pretraining [66.89012795621349]
  Large Language Models (LLMs) often struggle with the deep, long-horizon reasoning required for complex software engineering.<n>We propose a novel paradigm: understanding via reconstruction.<n>We introduce a framework that synthesizes latent agentic trajectories using a multi-agent simulation.
  arXiv  Detail & Related papers  (2026-03-11T09:23:20Z)
• ICLF: An Immersive Code Learning Framework based on Git for Teaching and Evaluating Student Programming Projects [2.030202554182725]
  The Immersive Code Learning Framework (ICLF) is a scalable Git-based organizational pipeline for managing and evaluating student programming projects.<n>Students begin with an existing code base, a practice that is crucial for mirroring real-world software development.<n>Students are invited collaborators on private forks of this intermediate repository, possibly updated throughout the semester whenever the teacher changes the parent repository.
  arXiv  Detail & Related papers  (2026-01-21T09:39:17Z)
• A Causal Perspective on Measuring, Explaining and Mitigating Smells in LLM-Generated Code [49.09545217453401]
  Propensity Smelly Score (PSC) is a metric that estimates the likelihood of generating particular smell types.<n>We identify how generation strategy, model size, model architecture and prompt formulation shape the structural properties of generated code.<n> PSC helps developers interpret model behavior and assess code quality, providing evidence that smell propensity signals can support human judgement.
  arXiv  Detail & Related papers  (2025-11-19T19:18:28Z)
• Evaluating Software Process Models for Multi-Agent Class-Level Code Generation [5.545076518491288]
  Large Language Models (LLMs) are increasingly used to automate software development.<n>This work examines how process structure and role shape multi-agent specialization for class-level code generation.
  arXiv  Detail & Related papers  (2025-11-12T22:53:12Z)
• Modeling Student Learning with 3.8 Million Program Traces [52.153493498021895]
  We introduce a dataset of over 3.8 million programming reasoning traces from users of Pencil Code.<n>We find that models trained on real traces are stronger at modeling diverse student behavior.<n>We show that we can help students recover from mistakes by steering code generation models to identify a sequence of edits that will result in more correct code.
  arXiv  Detail & Related papers  (2025-10-06T17:37:17Z)
• Training Language Models to Generate Quality Code with Program Analysis Feedback [66.0854002147103]
  Code generation with large language models (LLMs) is increasingly adopted in production but fails to ensure code quality.<n>We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code.
  arXiv  Detail & Related papers  (2025-05-28T17:57:47Z)
• RePurr: Automated Repair of Block-Based Learners' Programs [10.540099048026496]
  We introduce the first APR approach for Scratch based on evolutionary search.<n>Our RePurr prototype includes novel refinements of fault localization to improve the guidance of test suites.<n> Empirical evaluation on a set of real learners' programs confirms the anticipated challenges.
  arXiv  Detail & Related papers  (2025-04-16T19:22:51Z)
• ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models [81.12673534903979]
  Tool learning has emerged as a crucial capability for large language models (LLMs) to solve complex real-world tasks through interaction with external tools.<n>We propose ToolCoder, a novel framework that reformulates tool learning as a code generation task.
  arXiv  Detail & Related papers  (2025-02-17T03:42:28Z)
• Teaching Well-Structured Code: A Literature Review of Instructional Approaches [2.389598109913754]
  This systematic literature review identifies existing instructional approaches, their objectives, and the strategies used for measuring their effectiveness.<n>We classified these studies into three categories: (1) studies focused on developing or evaluating automated tools and their usage, (2) studies discussing other instructional materials, and (3) studies discussing how to integrate code structure into the curriculum through a holistic approach to course design to support code quality.
  arXiv  Detail & Related papers  (2025-02-16T18:51:22Z)
• Can Large Language Models Understand Intermediate Representations in Compilers? [17.033963652676164]
  Intermediate Representations (IRs) play a critical role in compiler design and program analysis.<n>We evaluate the capabilities of six state-of-the-art Large Language Models (LLMs) in understanding IRs.
  arXiv  Detail & Related papers  (2025-02-07T17:23:48Z)
• LLM-based Cognitive Models of Students with Misconceptions [55.29525439159345]
  This paper investigates whether Large Language Models (LLMs) can be instruction-tuned to meet this dual requirement. We introduce MalAlgoPy, a novel Python library that generates datasets reflecting authentic student solution patterns. Our insights enhance our understanding of AI-based student models and pave the way for effective adaptive learning systems.
  arXiv  Detail & Related papers  (2024-10-16T06:51:09Z)
• CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update [69.59482029810198]
  CLOVA is a Closed-Loop Visual Assistant that operates within a framework encompassing inference, reflection, and learning phases. Results demonstrate that CLOVA surpasses existing tool-usage methods by 5% in visual question answering and multiple-image reasoning, by 10% in knowledge tagging, and by 20% in image editing.
  arXiv  Detail & Related papers  (2023-12-18T03:34:07Z)
• Enhancing Automated Program Repair through Fine-tuning and Prompt Engineering [2.3826139428423576]
  Sequence-to-sequence models have been used to transform erroneous programs into correct ones when trained with a large enough dataset. Some recent studies demonstrated strong empirical evidence that code review could improve the program repair further. We investigate if this inherent knowledge of PL and NL can be utilized to improve automated program repair.
  arXiv  Detail & Related papers  (2023-04-16T17:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.