What is a "bug"? On subjectivity, epistemic power, and implications for software research

• URL: http://arxiv.org/abs/2402.08165v1
• Date: Tue, 13 Feb 2024 01:52:42 GMT
• Title: What is a "bug"? On subjectivity, epistemic power, and implications for software research
• Authors: David Gray Widder and Claire Le Goues
• Abstract summary: "Bug" has been a colloquialism for an engineering "defect" at least since the 1870s. Most modern software-oriented definitions speak to a disconnect between what a developer intended and what a program actually does. "Finding bugs is easy" begins by saying "bug patterns are code that are often errors"
• Score: 8.116831482130555
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Considerable effort in software research and practice is spent on bugs. Finding, reporting, tracking, triaging, attempting to fix them automatically, detecting "bug smells" -these comprise a substantial portion of large projects' time and development cost, and are of significant interest to researchers in Software Engineering, Programming Languages, and beyond. But, what is a bug, exactly? While segmentation faults rarely spark joy, most bugs are not so clear cut. Per the Oxford English Dictionary, the word "bug" has been a colloquialism for an engineering "defect" at least since the 1870s. Most modern software-oriented definitions speak to a disconnect between what a developer intended and what a program actually does. Formal verification, from its inception, has developed means to identify deviations from a formal specification, expected to more or less fully encode desired behavior. However, software is rarely accompanied by full and formal specifications, and this intention is instead treated as implicit or partially-documented at best. The International Software Testing Qualifications board writes: "A human being can make an error (mistake), which produces a defect (fault, bug) in the program code, or in a document. If a defect in code is executed, the system may fail to do what it should do (or do something it shouldn't), causing a failure. Defects may result in failures, but not all [do]". Most sources forsake this precision. The influential paper "Finding bugs is easy" begins by saying "bug patterns are code idioms that are often errors"-with no particular elaboration. Other work relies on imperfect practical proxies for specifications. For example, in automatic program repair research, a bug corresponds to a failing test case: when the test passes, the bug is considered fixed. However, when we interrogate fairly straightforward definitions, they start to break down...

Related papers

• Automated Bug Generation in the era of Large Language Models [6.0770779409377775]
  BugFarm transforms arbitrary code into multiple complex bugs. A comprehensive evaluation of 435k+ bugs from over 1.9M mutants generated by BUGFARM.
  arXiv  Detail & Related papers  (2023-10-03T20:01:51Z)
• Bugsplainer: Leveraging Code Structures to Explain Software Bugs with Neural Machine Translation [4.519754139322585]
  Bugsplainer generates natural language explanations for software bugs by learning from a large corpus of bug-fix commits. Bugsplainer leverages code structures to reason about a bug and employs the fine-tuned version of a text generation model, CodeT5.
  arXiv  Detail & Related papers  (2023-08-23T17:35:16Z)
• Large Language Models of Code Fail at Completing Code with Potential Bugs [30.80172644795715]
  We study the buggy-code completion problem inspired by real-time code suggestion. We find that the presence of potential bugs significantly degrades the generation performance of the high-performing Code-LLMs.
  arXiv  Detail & Related papers  (2023-06-06T06:35:27Z)
• Teaching Large Language Models to Self-Debug [62.424077000154945]
  Large language models (LLMs) have achieved impressive performance on code generation. We propose Self- Debugging, which teaches a large language model to debug its predicted program via few-shot demonstrations.
  arXiv  Detail & Related papers  (2023-04-11T10:43:43Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)
• Explaining Software Bugs Leveraging Code Structures in Neural Machine Translation [5.079750706023254]
  Bugsplainer generates natural language explanations for software bugs by learning from a large corpus of bug-fix commits. Our evaluation using three performance metrics shows that Bugsplainer can generate understandable and good explanations according to Google's standard. We also conduct a developer study involving 20 participants where the explanations from Bugsplainer were found to be more accurate, more precise, more concise and more useful than the baselines.
  arXiv  Detail & Related papers  (2022-12-08T22:19:45Z)
• Using Developer Discussions to Guide Fixing Bugs in Software [51.00904399653609]
  We propose using bug report discussions, which are available before the task is performed and are also naturally occurring, avoiding the need for additional information from developers. We demonstrate that various forms of natural language context derived from such discussions can aid bug-fixing, even leading to improved performance over using commit messages corresponding to the oracle bug-fixing commits.
  arXiv  Detail & Related papers  (2022-11-11T16:37:33Z)
• BigIssue: A Realistic Bug Localization Benchmark [89.8240118116093]
  BigIssue is a benchmark for realistic bug localization. We provide a general benchmark with a diversity of real and synthetic Java bugs. We hope to advance the state of the art in bug localization, in turn improving APR performance and increasing its applicability to the modern development cycle.
  arXiv  Detail & Related papers  (2022-07-21T20:17:53Z)
• Fault-Aware Neural Code Rankers [64.41888054066861]
  We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it. Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
  arXiv  Detail & Related papers  (2022-06-04T22:01:05Z)
• Identifying Bug Patterns in Quantum Programs [4.282118876884235]
  Bug patterns are erroneous code idioms or bad coding practices that have been proved to fail time and time again. This paper identifies and categorizes some bug patterns in the quantum programming language Qiskit.
  arXiv  Detail & Related papers  (2021-03-16T13:43:45Z)
• On the Robustness of Language Encoders against Grammatical Errors [66.05648604987479]
  We collect real grammatical errors from non-native speakers and conduct adversarial attacks to simulate these errors on clean text data. Results confirm that the performance of all tested models is affected but the degree of impact varies.
  arXiv  Detail & Related papers  (2020-05-12T11:01:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.