Flexible Control Flow Graph Alignment for Delivering Data-Driven Feedback to Novice Programming Learners

• URL: http://arxiv.org/abs/2401.01416v1
• Date: Tue, 2 Jan 2024 19:56:50 GMT
• Title: Flexible Control Flow Graph Alignment for Delivering Data-Driven Feedback to Novice Programming Learners
• Authors: Md Towhidul Absar Chowdhury, Maheen Riaz Contractor, Carlos R. Rivero
• Abstract summary: We present several modifications to CLARA, a data-driven automated repair approach that is open source. We extend CLARA's abstract syntax tree processor to handle common introductory programming constructs. We modify an incorrect program's control flow graph to match the correct programs to apply CLARA's original repair process.
• Score: 0.847136673632881
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Supporting learners in introductory programming assignments at scale is a necessity. This support includes automated feedback on what learners did incorrectly. Existing approaches cast the problem as automatically repairing learners' incorrect programs extrapolating the data from an existing correct program from other learners. However, such approaches are limited because they only compare programs with similar control flow and order of statements. A potentially valuable set of repair feedback from flexible comparisons is thus missing. In this paper, we present several modifications to CLARA, a data-driven automated repair approach that is open source, to deal with real-world introductory programs. We extend CLARA's abstract syntax tree processor to handle common introductory programming constructs. Additionally, we propose a flexible alignment algorithm over control flow graphs where we enrich nodes with semantic annotations extracted from programs using operations and calls. Using this alignment, we modify an incorrect program's control flow graph to match the correct programs to apply CLARA's original repair process. We evaluate our approach against a baseline on the twenty most popular programming problems in Codeforces. Our results indicate that flexible alignment has a significantly higher percentage of successful repairs at 46% compared to 5% for baseline CLARA. Our implementation is available at https://github.com/towhidabsar/clara.

Related papers

• A Novel Approach for Automatic Program Repair using Round-Trip Translation with Large Language Models [50.86686630756207]
  Research shows that grammatical mistakes in a sentence can be corrected by translating it to another language and back. Current generative models for Automatic Program Repair (APR) are pre-trained on source code and fine-tuned for repair. This paper proposes bypassing the fine-tuning step and using Round-Trip Translation (RTT): translation of code from one programming language to another programming or natural language, and back.
  arXiv  Detail & Related papers  (2024-01-15T22:36:31Z)
• How Helpful do Novice Programmers Find the Feedback of an Automated Repair Tool? [1.2990666399718034]
  We describe our experience of using CLARA, an automated repair tool, to provide feedback to novices. First, we extended CLARA to support a larger subset of the Python language, before integrating it with the Jupyter Notebooks used for our programming exercises. We found that novices often struggled to understand the proposed repairs, echoing the well-known challenge to understand compiler/interpreter messages.
  arXiv  Detail & Related papers  (2023-10-02T07:45:56Z)
• Neural Program Repair with Program Dependence Analysis and Effective Filter Mechanism [37.70518599085677]
  We present a novel neural program repair framework called approach, which adapts the general pre-trained language model for fixing single-line Java bugs. We make the first attempt to use program slicing to extract contextual information directly related to the given buggy statement as repair ingredients from the corresponding program dependence graph. We demonstrate the effectiveness of approach on five benchmarks when compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-05-16T09:43:04Z)
• RunBugRun -- An Executable Dataset for Automated Program Repair [15.670905650869704]
  We present a fully executable dataset of 450,000 small buggy/fixed program pairs originally submitted to programming competition websites. We provide infrastructure to compile, safely execute and test programs as well as fine-grained bug-type labels.
  arXiv  Detail & Related papers  (2023-04-03T16:02:00Z)
• The Wisdom of Hindsight Makes Language Models Better Instruction Followers [84.9120606803906]
  Reinforcement learning has seen wide success in finetuning large language models to better align with instructions via human feedback. In this paper, we consider an alternative approach: converting feedback to instruction by relabeling the original one and training the model for better alignment in a supervised manner. We propose Hindsight Instruction Relabeling (HIR), a novel algorithm for aligning language models with instructions.
  arXiv  Detail & Related papers  (2023-02-10T12:16:38Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• NAPG: Non-Autoregressive Program Generation for Hybrid Tabular-Textual Question Answering [52.10214317661547]
  Current numerical reasoning methods autoregressively decode program sequences. The accuracy of program generation drops sharply as the decoding steps unfold due to error propagation. In this paper, we propose a non-autoregressive program generation framework.
  arXiv  Detail & Related papers  (2022-11-07T11:25:21Z)
• Learning from Self-Sampled Correct and Partially-Correct Programs [96.66452896657991]
  We propose to let the model perform sampling during training and learn from both self-sampled fully-correct programs and partially-correct programs. We show that our use of self-sampled correct and partially-correct programs can benefit learning and help guide the sampling process. Our proposed method improves the pass@k performance by 3.1% to 12.3% compared to learning from a single reference program with MLE.
  arXiv  Detail & Related papers  (2022-05-28T03:31:07Z)
• Leveraging Causal Inference for Explainable Automatic Program Repair [24.146216081282798]
  This paper presents an interpretable approach for program repair based on sequence-to-sequence models with causal inference. Our method is called CPR, short for causal program repair. Experiments on four programming languages show that CPR can generate causal graphs for reasonable interpretations.
  arXiv  Detail & Related papers  (2022-05-26T13:25:33Z)
• Graph-based, Self-Supervised Program Repair from Diagnostic Feedback [108.48853808418725]
  We introduce a program-feedback graph, which connects symbols relevant to program repair in source code and diagnostic feedback. We then apply a graph neural network on top to model the reasoning process. We present a self-supervised learning paradigm for program repair that leverages unlabeled programs available online.
  arXiv  Detail & Related papers  (2020-05-20T07:24:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.