TestSpark: IntelliJ IDEA's Ultimate Test Generation Companion

• URL: http://arxiv.org/abs/2401.06580v1
• Date: Fri, 12 Jan 2024 13:53:57 GMT
• Title: TestSpark: IntelliJ IDEA's Ultimate Test Generation Companion
• Authors: Arkadii Sapozhnikov, Mitchell Olsthoorn, Annibale Panichella, Vladimir Kovalenko, Pouria Derakhshanfar
• Abstract summary: This paper introduces TestSpark, a plugin for IntelliJ IDEA that enables users to generate unit tests with only a few clicks. TestSpark also allows users to easily modify and run each generated test and integrate them into the project workflow.
• Score: 15.13443954421825
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Writing software tests is laborious and time-consuming. To address this, prior studies introduced various automated test-generation techniques. A well-explored research direction in this field is unit test generation, wherein artificial intelligence (AI) techniques create tests for a method/class under test. While many of these techniques have primarily found applications in a research context, existing tools (e.g., EvoSuite, Randoop, and AthenaTest) are not user-friendly and are tailored to a single technique. This paper introduces TestSpark, a plugin for IntelliJ IDEA that enables users to generate unit tests with only a few clicks directly within their Integrated Development Environment (IDE). Furthermore, TestSpark also allows users to easily modify and run each generated test and integrate them into the project workflow. TestSpark leverages the advances of search-based test generation tools, and it introduces a technique to generate unit tests using Large Language Models (LLMs) by creating a feedback cycle between the IDE and the LLM. Since TestSpark is an open-source (https://github.com/JetBrains-Research/TestSpark), extendable, and well-documented tool, it is possible to add new test generation methods into the plugin with the minimum effort. This paper also explains our future studies related to TestSpark and our preliminary results. Demo video: https://youtu.be/0F4PrxWfiXo

Related papers

• Multi-language Unit Test Generation using LLMs [6.259245181881262]
  We describe a generic pipeline that incorporates static analysis to guide LLMs in generating compilable and high-coverage test cases. We show how the pipeline can be applied to different programming languages, specifically Java and Python, and to complex software requiring environment mocking. Our results demonstrate that LLM-based test generation, when guided by static analysis, can be competitive with, and even outperform, state-of-the-art test-generation techniques in coverage achieved.
  arXiv  Detail & Related papers  (2024-09-04T21:46:18Z)
• A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites [1.4563527353943984]
  Large Language Models (LLMs) have been applied to various aspects of software development. We present AgoneTest: an automated system for generating test suites for Java projects.
  arXiv  Detail & Related papers  (2024-08-14T23:02:16Z)
• Automated Unit Test Improvement using Large Language Models at Meta [44.87533111512982]
  This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms.
  arXiv  Detail & Related papers  (2024-02-14T13:43:14Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• PyTester: Deep Reinforcement Learning for Text-to-Testcase Generation [20.441921569948562]
  Test-driven development (TDD) mandates writing test cases based on requirements before writing the actual code. While writing test cases is the centerpiece of TDD, it is time-consuming, expensive, and often shunned by developers. We introduce PyTester, a Text-to-Testcase generation approach that can automatically generate correct, executable, complete, and effective test cases.
  arXiv  Detail & Related papers  (2024-01-15T10:21:58Z)
• Leveraging Large Language Models to Improve REST API Testing [51.284096009803406]
  RESTGPT takes as input an API specification, extracts machine-interpretable rules, and generates example parameter values from natural-language descriptions in the specification. Our evaluations indicate that RESTGPT outperforms existing techniques in both rule extraction and value generation.
  arXiv  Detail & Related papers  (2023-12-01T19:53:23Z)
• Do Automatic Test Generation Tools Generate Flaky Tests? [12.813573907094074]
  The prevalence and nature of flaky tests produced by test generation tools remain largely unknown. We generate tests using EvoSuite (Java) and Pynguin (Python) and execute each test 200 times. Our results show that flakiness is at least as common in generated tests as in developer-written tests.
  arXiv  Detail & Related papers  (2023-10-08T16:44:27Z)
• Towards Automatic Generation of Amplified Regression Test Oracles [44.45138073080198]
  We propose a test oracle derivation approach to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour.
  arXiv  Detail & Related papers  (2023-07-28T12:38:44Z)
• Learning Deep Semantics for Test Completion [46.842174440120196]
  We formalize the novel task of test completion to automatically complete the next statement in a test method based on the context of prior statements and the code under test. We develop TeCo -- a deep learning model using code semantics for test completion.
  arXiv  Detail & Related papers  (2023-02-20T18:53:56Z)
• BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models [73.29106813131818]
  bias testing is currently cumbersome since the test sentences are generated from a limited set of manual templates or need expensive crowd-sourcing. We propose using ChatGPT for the controllable generation of test sentences, given any arbitrary user-specified combination of social groups and attributes. We present an open-source comprehensive bias testing framework (BiasTestGPT), hosted on HuggingFace, that can be plugged into any open-source PLM for bias testing.
  arXiv  Detail & Related papers  (2023-02-14T22:07:57Z)
• Automated Support for Unit Test Generation: A Tutorial Book Chapter [21.716667622896193]
  Unit testing is a stage of testing where the smallest segment of code that can be tested in isolation from the rest of the system is tested. Unit tests are typically written as executable code, often in a format provided by a unit testing framework such as pytest for Python. This chapter introduces the concept of search-based unit test generation.
  arXiv  Detail & Related papers  (2021-10-26T11:13:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.