A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools

• URL: http://arxiv.org/abs/2403.05939v2
• Date: Fri, 23 Aug 2024 10:04:27 GMT
• Title: A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools
• Authors: Pouria Alikhanifard, Nikolaos Tsantalis,
• Abstract summary: Syntax Tree (AST) diff tools were developed to overcome the limitations of line-based diff tools, which are used by the majority of developers. We propose a novel AST diff tool based on RefactoringMiner that resolves all aforementioned limitations. Our tool achieved a considerably higher precision and recall, especially for commits, with an execution time that is comparable with incompatible tools.
• Score: 2.0625936401496237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software undergoes constant changes to support new requirements, address bugs, enhance performance, and ensure maintainability. Thus, developers spend a great portion of their workday trying to understand and review the code changes of their teammates. Abstract Syntax Tree (AST) diff tools were developed to overcome the limitations of line-based diff tools, which are used by the majority of developers. Despite the notable improvements brought by AST diff tools in understanding complex changes, they still suffer from serious limitations, such as (1) lacking multi-mapping support, (2) matching semantically incompatible AST nodes, (3) ignoring language clues to guide the matching process, (4) lacking refactoring awareness, and (5) lacking commit-level diff support. We propose a novel AST diff tool based on RefactoringMiner that resolves all aforementioned limitations. First, we improved RefactoringMiner to increase its statement mapping accuracy, and then we developed an algorithm that generates AST diff for a given commit or pull request based on the refactoring instances and pairs of matched program element declarations provided by RefactoringMiner. To evaluate the accuracy of our tool and compare it with the state-of-the-art tools, we created the first benchmark of AST node mappings, including 800 bug-fixing commits and 188 refactoring commits. Our evaluation showed that our tool achieved a considerably higher precision and recall, especially for refactoring commits, with an execution time that is comparable with that of the faster tools.

Related papers

• Semistructured Merge with Language-Specific Syntactic Separators [1.0999592665107416]
  We propose a tool that uses language-specific syntactic separators to infer structure without parsing. Our tool shows significant improvements over unstructured tools widely used in practice.
  arXiv  Detail & Related papers  (2024-07-26T17:40:29Z)
• Automatic Build Repair for Test Cases using Incompatible Java Versions [7.4881561767138365]
  We introduce an approach to repair test cases of Java projects by performing dependency minimization. Unlike existing state-of-the-art techniques, our approach performs at source-level, which allows compile-time errors to be fixed.
  arXiv  Detail & Related papers  (2024-04-27T07:55:52Z)
• Contrastive Instruction Tuning [61.97704869248903]
  We propose Contrastive Instruction Tuning to maximize the similarity between semantically equivalent instruction-instance pairs. Experiments on the PromptBench benchmark show that CoIN consistently improves LLMs' robustness to unseen instructions with variations across character, word, sentence, and semantic levels by an average of +2.5% in accuracy.
  arXiv  Detail & Related papers  (2024-02-17T00:09:32Z)
• ControlLLM: Augment Language Models with Tools by Searching on Graphs [97.62758830255002]
  We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving real-world tasks. Our framework comprises three key components: (1) a textittask decomposer that breaks down a complex task into clear subtasks with well-defined inputs and outputs; (2) a textitThoughts-on-Graph (ToG) paradigm that searches the optimal solution path on a pre-built tool graph; and (3) an textitexecution engine with a rich toolbox that interprets the solution path and runs the
  arXiv  Detail & Related papers  (2023-10-26T21:57:21Z)
• RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair [75.40584530380589]
  We propose a novel Retrieval-Augmented Patch Generation framework (RAP-Gen) RAP-Gen explicitly leveraging relevant fix patterns retrieved from a list of previous bug-fix pairs. We evaluate RAP-Gen on three benchmarks in two programming languages, including the TFix benchmark in JavaScript, and Code Refinement and Defects4J benchmarks in Java.
  arXiv  Detail & Related papers  (2023-09-12T08:52:56Z)
• Do code refactorings influence the merge effort? [80.1936417993664]
  Multiple contributors frequently change the source code in parallel to implement new features, fix bugs, existing code, and make other changes. These simultaneous changes need to be merged into the same version of the source code. Studies show that 10 to 20 percent of all merge attempts result in conflicts, which require the manual developer's intervention to complete the process.
  arXiv  Detail & Related papers  (2023-05-10T13:24:59Z)
• ART: Automatic multi-step reasoning and tool-use for large language models [105.57550426609396]
  Large language models (LLMs) can perform complex reasoning in few- and zero-shot settings. Each reasoning step can rely on external tools to support computation beyond the core LLM capabilities. We introduce Automatic Reasoning and Tool-use (ART), a framework that uses frozen LLMs to automatically generate intermediate reasoning steps as a program.
  arXiv  Detail & Related papers  (2023-03-16T01:04:45Z)
• Accelerating DETR Convergence via Semantic-Aligned Matching [50.3633635846255]
  This paper presents SAM-DETR, a Semantic-Aligned-Matching DETR that greatly accelerates DETR's convergence without sacrificing its accuracy. It explicitly searches salient points with the most discriminative features for semantic-aligned matching, which further speeds up the convergence and boosts detection accuracy as well.
  arXiv  Detail & Related papers  (2022-03-14T06:50:51Z)
• Generating Bug-Fixes Using Pretrained Transformers [11.012132897417592]
  We introduce a data-driven program repair approach which learns to detect and fix bugs in Java methods mined from real-world GitHub. We show that pretraining on source code programs improves the number of patches found by 33% as compared to supervised training from scratch. We refine the standard accuracy evaluation metric into non-deletion and deletion-only fixes, and show that our best model generates 75% more non-deletion fixes than the previous state of the art.
  arXiv  Detail & Related papers  (2021-04-16T05:27:04Z)
• How We Refactor and How We Document it? On the Use of Supervised Machine Learning Algorithms to Classify Refactoring Documentation [25.626914797750487]
  Refactoring is the art of improving the design of a system without altering its external behavior. This study categorizes commits into 3 categories, namely, Internal QA, External QA, and Code Smell Resolution, along with the traditional BugFix and Functional categories. To better understand our classification results, we analyzed commit messages to extract patterns that developers regularly use to describe their smells.
  arXiv  Detail & Related papers  (2020-10-26T20:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.