A Survey on What Developers Think About Testing

• URL: http://arxiv.org/abs/2309.01154v1
• Date: Sun, 3 Sep 2023 12:18:41 GMT
• Title: A Survey on What Developers Think About Testing
• Authors: Philipp Straubinger, Gordon Fraser
• Abstract summary: We conducted a comprehensive survey with 21 questions aimed at assessing developers' current engagement with testing. We uncover reasons that positively and negatively impact developers' motivation to test. One approach emerging from the responses to mitigate these negative factors is by providing better recognition for developers' testing efforts.
• Score: 13.086283144520513
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Software is infamous for its poor quality and frequent occurrence of bugs. While there is no doubt that thorough testing is an appropriate answer to ensure sufficient quality, the poor state of software generally suggests that developers may not always engage as thoroughly with testing as they should. This observation aligns with the prevailing belief that developers simply do not like writing tests. In order to determine the truth of this belief, we conducted a comprehensive survey with 21 questions aimed at (1) assessing developers' current engagement with testing and (2) identifying factors influencing their inclination toward testing; that is, whether they would actually like to test more but are inhibited by their work environment, or whether they would really prefer to test even less if given the choice. Drawing on 284 responses from professional software developers, we uncover reasons that positively and negatively impact developers' motivation to test. Notably, reasons for motivation to write more tests encompass not only a general pursuit of software quality but also personal satisfaction. However, developers nevertheless perceive testing as mundane and tend to prioritize other tasks. One approach emerging from the responses to mitigate these negative factors is by providing better recognition for developers' testing efforts.

Related papers

• Do Test and Environmental Complexity Increase Flakiness? An Empirical Study of SAP HANA [47.29324864511411]
  Flaky tests fail seemingly at random without changes to the code. We study characteristics of tests and the test environment that potentially impact test flakiness.
  arXiv  Detail & Related papers  (2024-09-16T07:52:09Z)
• Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests [4.574205608859157]
  We introduce UTGen, which combines search-based software testing and large language models to enhance the understandability of automatically generated test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases.
  arXiv  Detail & Related papers  (2024-08-21T15:35:34Z)
• Leveraging Large Language Models for Efficient Failure Analysis in Game Development [47.618236610219554]
  This paper proposes a new approach to automatically identify which change in the code caused a test to fail. The method leverages Large Language Models (LLMs) to associate error messages with the corresponding code changes causing the failure. Our approach reaches an accuracy of 71% in our newly created dataset, which comprises issues reported by developers at EA over a period of one year.
  arXiv  Detail & Related papers  (2024-06-11T09:21:50Z)
• Elevating Software Quality in Agile Environments: The Role of Testing Professionals in Unit Testing [0.0]
  Testing is an essential quality activity in the software development process. This paper explores the participation of test engineers in unit testing within an industrial context.
  arXiv  Detail & Related papers  (2024-03-20T00:41:49Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• Improving Testing Behavior by Gamifying IntelliJ [13.086283144520513]
  We introduce IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated Development Environment. IntelliGame rewards developers for positive testing behavior using a multi-level achievement system. A controlled experiment with 49 participants reveals substantial differences in the testing behavior triggered by IntelliGame.
  arXiv  Detail & Related papers  (2023-10-17T11:40:55Z)
• Software Testing and Code Refactoring: A Survey with Practitioners [3.977213079821398]
  This study aims to explore how software testing professionals deal with code to understand the benefits and limitations of this practice in the context of software testing. We concluded that in the context of software testing, offers several benefits, such as supporting the maintenance of automated tests and improving the performance of the testing team. Our study raises discussions on the importance of having testing professionals implement in the code of automated tests, allowing them to improve their coding abilities.
  arXiv  Detail & Related papers  (2023-10-03T01:07:39Z)
• Test case quality: an empirical study on belief and evidence [8.475270520855332]
  We investigate eight hypotheses regarding what constitutes a good test case. Despite our best efforts, we were unable to find evidence that supports these beliefs.
  arXiv  Detail & Related papers  (2023-07-12T19:02:48Z)
• SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video Games Using Risk Based Testing and Machine Learning [62.997667081978825]
  Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems. We present SUPERNOVA, a system responsible for test selection and defect prevention while also functioning as an automation hub. The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title.
  arXiv  Detail & Related papers  (2022-03-10T00:47:46Z)
• Beyond Accuracy: Behavioral Testing of NLP models with CheckList [66.42971817954806]
  CheckList is a task-agnostic methodology for testing NLP models. CheckList includes a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation. In a user study, NLP practitioners with CheckList created twice as many tests, and found almost three times as many bugs as users without it.
  arXiv  Detail & Related papers  (2020-05-08T15:48:31Z)
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design [63.48989885374238]
  When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testing algorithms that can operate in a noisy setting.
  arXiv  Detail & Related papers  (2020-04-26T23:41:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.