Explaining Software Bugs Leveraging Code Structures in Neural Machine
Translation
- URL: http://arxiv.org/abs/2212.04584v4
- Date: Tue, 25 Jul 2023 05:58:33 GMT
- Title: Explaining Software Bugs Leveraging Code Structures in Neural Machine
Translation
- Authors: Parvez Mahbub, Ohiduzzaman Shuvo, Mohammad Masudur Rahman
- Abstract summary: Bugsplainer generates natural language explanations for software bugs by learning from a large corpus of bug-fix commits.
Our evaluation using three performance metrics shows that Bugsplainer can generate understandable and good explanations according to Google's standard.
We also conduct a developer study involving 20 participants where the explanations from Bugsplainer were found to be more accurate, more precise, more concise and more useful than the baselines.
- Score: 5.079750706023254
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Software bugs claim approximately 50% of development time and cost the global
economy billions of dollars. Once a bug is reported, the assigned developer
attempts to identify and understand the source code responsible for the bug and
then corrects the code. Over the last five decades, there has been significant
research on automatically finding or correcting software bugs. However, there
has been little research on automatically explaining the bugs to the
developers, which is essential but a highly challenging task. In this paper, we
propose Bugsplainer, a transformer-based generative model, that generates
natural language explanations for software bugs by learning from a large corpus
of bug-fix commits. Bugsplainer can leverage structural information and buggy
patterns from the source code to generate an explanation for a bug. Our
evaluation using three performance metrics shows that Bugsplainer can generate
understandable and good explanations according to Google's standard, and can
outperform multiple baselines from the literature. We also conduct a developer
study involving 20 participants where the explanations from Bugsplainer were
found to be more accurate, more precise, more concise and more useful than the
baselines.
Related papers
- What's Wrong with Your Code Generated by Large Language Models? An Extensive Study [80.18342600996601]
Large language models (LLMs) produce code that is shorter yet more complicated as compared to canonical solutions.
We develop a taxonomy of bugs for incorrect codes that includes three categories and 12 sub-categories, and analyze the root cause for common bug types.
We propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code based on bug types and compiler feedback.
arXiv Detail & Related papers (2024-07-08T17:27:17Z) - What is a "bug"? On subjectivity, epistemic power, and implications for
software research [8.116831482130555]
"Bug" has been a colloquialism for an engineering "defect" at least since the 1870s.
Most modern software-oriented definitions speak to a disconnect between what a developer intended and what a program actually does.
"Finding bugs is easy" begins by saying "bug patterns are code that are often errors"
arXiv Detail & Related papers (2024-02-13T01:52:42Z) - DebugBench: Evaluating Debugging Capability of Large Language Models [80.73121177868357]
DebugBench is a benchmark for Large Language Models (LLMs)
It covers four major bug categories and 18 minor types in C++, Java, and Python.
We evaluate two commercial and four open-source models in a zero-shot scenario.
arXiv Detail & Related papers (2024-01-09T15:46:38Z) - Automated Bug Generation in the era of Large Language Models [6.519768481767584]
We propose BugFarm, to transform arbitrary code into multiple complex bugs.
BugFarm generates bugs that are hard to detect by learning-based bug prediction approaches and hard to repair by SOTA learning-based program repair technique.
arXiv Detail & Related papers (2023-10-03T20:01:51Z) - Bugsplainer: Leveraging Code Structures to Explain Software Bugs with
Neural Machine Translation [4.519754139322585]
Bugsplainer generates natural language explanations for software bugs by learning from a large corpus of bug-fix commits.
Bugsplainer leverages code structures to reason about a bug and employs the fine-tuned version of a text generation model, CodeT5.
arXiv Detail & Related papers (2023-08-23T17:35:16Z) - Large Language Models of Code Fail at Completing Code with Potential
Bugs [30.80172644795715]
We study the buggy-code completion problem inspired by real-time code suggestion.
We find that the presence of potential bugs significantly degrades the generation performance of the high-performing Code-LLMs.
arXiv Detail & Related papers (2023-06-06T06:35:27Z) - Teaching Large Language Models to Self-Debug [62.424077000154945]
Large language models (LLMs) have achieved impressive performance on code generation.
We propose Self- Debugging, which teaches a large language model to debug its predicted program via few-shot demonstrations.
arXiv Detail & Related papers (2023-04-11T10:43:43Z) - Using Developer Discussions to Guide Fixing Bugs in Software [51.00904399653609]
We propose using bug report discussions, which are available before the task is performed and are also naturally occurring, avoiding the need for additional information from developers.
We demonstrate that various forms of natural language context derived from such discussions can aid bug-fixing, even leading to improved performance over using commit messages corresponding to the oracle bug-fixing commits.
arXiv Detail & Related papers (2022-11-11T16:37:33Z) - BigIssue: A Realistic Bug Localization Benchmark [89.8240118116093]
BigIssue is a benchmark for realistic bug localization.
We provide a general benchmark with a diversity of real and synthetic Java bugs.
We hope to advance the state of the art in bug localization, in turn improving APR performance and increasing its applicability to the modern development cycle.
arXiv Detail & Related papers (2022-07-21T20:17:53Z) - DapStep: Deep Assignee Prediction for Stack Trace Error rePresentation [61.99379022383108]
We propose new deep learning models to solve the bug triage problem.
The models are based on a bidirectional recurrent neural network with attention and on a convolutional neural network.
To improve the quality of ranking, we propose using additional information from version control system annotations.
arXiv Detail & Related papers (2022-01-14T00:16:57Z) - Advaita: Bug Duplicity Detection System [1.9624064951902522]
Duplicate bugs rate (% of duplicate bugs) are in the range from single digit (1 to 9%) to double digits (40%) based on the product maturity, size of the code and number of engineers working on the project.
Detecting duplicity deals with identifying whether any two bugs convey the same meaning.
This approach considers multiple sets of features viz. basic text statistical features, semantic features and contextual features.
arXiv Detail & Related papers (2020-01-24T04:48:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.