On the Energy Consumption of Test Generation

• URL: http://arxiv.org/abs/2501.09657v1
• Date: Thu, 16 Jan 2025 16:52:55 GMT
• Title: On the Energy Consumption of Test Generation
• Abstract summary: We measure the energy consumed by EvoSuite when generating tests and the energy consumed in the execution of the test cases generated.<n>Results show that the different test generation algorithms consumed different amounts of energy, in particular on classes with high cyclomatic complexity.
• Score: 0.8749675983608172
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Research in the area of automated test generation has seen remarkable progress in recent years, resulting in several approaches and tools for effective and efficient generation of test cases. In particular, the EvoSuite tool has been at the forefront of this progress embodying various algorithms for automated test generation of Java programs. EvoSuite has been used to generate test cases for a wide variety of programs as well. While there are a number of empirical studies that report results on the effectiveness, in terms of code coverage and other related metrics, of the various test generation strategies and algorithms implemented in EvoSuite, there are no studies, to the best of our knowledge, on the energy consumption associated to the automated test generation. In this paper, we set out to investigate this aspect by measuring the energy consumed by EvoSuite when generating tests. We also measure the energy consumed in the execution of the test cases generated, comparing them with those manually written by developers. The results show that the different test generation algorithms consumed different amounts of energy, in particular on classes with high cyclomatic complexity. Furthermore, we also observe that manual tests tend to consume more energy as compared to automatically generated tests, without necessarily achieving higher code coverage. Our results also give insight into the methods that consume significantly higher levels of energy, indicating potential points of improvement both for EvoSuite as well as the different programs under test.

Related papers

• Testing with AI Agents: An Empirical Study of Test Generation Frequency, Quality, and Coverage [1.6782288954198918]
  This study presents an empirical analysis of test generation by agent-based coding tools using the AIDev dataset.<n>We extracted 2,232 commits containing test-related changes and investigated three aspects: the frequency of test additions, the structural characteristics of the generated tests, and their impact on code coverage.<n>Our findings reveal that (i) AI authored 16.4% of all commits adding tests in real-world repositories, (ii) AI-generated test methods exhibit distinct structural patterns, featuring longer code and a higher density of assertions, and (iii) AI-generated tests contribute to code coverage comparable to human-written tests
  arXiv  Detail & Related papers  (2026-03-14T03:01:44Z)
• Scaling Agentic Verifier for Competitive Coding [66.11758166379092]
  Large language models (LLMs) have demonstrated strong coding capabilities but still struggle to solve competitive programming problems correctly in a single attempt.<n>Execution-based re-ranking offers a promising test-time scaling strategy, yet existing methods are constrained by either difficult test case generation or inefficient random input sampling.<n>We propose Agentic Verifier, an execution-based agent that actively reasons about program behaviors and searches for highly discriminative test inputs.
  arXiv  Detail & Related papers  (2026-02-04T06:30:40Z)
• ATGen: Adversarial Reinforcement Learning for Test Case Generation [78.48498301767079]
  Large Language Models (LLMs) excel at code generation, yet their outputs often contain subtle bugs.<n>Existing test generation methods rely on static datasets.<n>We introduce ATGen, a framework that trains a test case generator via adversarial reinforcement learning.
  arXiv  Detail & Related papers  (2025-10-16T12:49:25Z)
• CodeChemist: Functional Knowledge Transfer for Low-Resource Code Generation via Test-Time Scaling [63.08126845138046]
  We present CodeChemist, a framework for test-time scaling that enables functional knowledge transfer from high-resource to low-resource PLs.<n>Our experiments show that CodeChemist outperforms existing test-time scaling approaches.
  arXiv  Detail & Related papers  (2025-10-01T04:33:53Z)
• Hamster: A Large-Scale Study and Characterization of Developer-Written Tests [44.65515600399573]
  We study developer-written tests for Java applications, covering 1.7 million test cases from open-source repositories.<n>Our results highlight that a vast majority of developer-written tests exhibit characteristics beyond the capabilities of current ATG tools.<n>We identify promising research directions that can help bridge the gap between current tool capabilities and more effective tool support for developer testing practices.
  arXiv  Detail & Related papers  (2025-09-30T13:08:23Z)
• Intention-Driven Generation of Project-Specific Test Cases [45.2380093475221]
  We propose IntentionTest, which generates project-specific tests given the description of validation intention.<n>We extensively evaluate IntentionTest against state-of-the-art baselines (DA, ChatTester, and EvoSuite) on 4,146 test cases from 13 open-source projects.
  arXiv  Detail & Related papers  (2025-07-28T08:35:04Z)
• From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking [48.90371827091671]
  AutoExperiment is a benchmark that evaluates AI agents' ability to implement and run machine learning experiments.<n>We evaluate state-of-the-art agents and find that performance degrades rapidly as $n$ increases.<n>Our findings highlight critical challenges in long-horizon code generation, context retrieval, and autonomous experiment execution.
  arXiv  Detail & Related papers  (2025-06-24T15:39:20Z)
• EvoGPT: Enhancing Test Suite Robustness via LLM-Based Generation and Genetic Optimization [11.050047263054985]
  Large Language Models (LLMs) have recently emerged as promising tools for automated unit test generation.<n>We introduce a hybrid framework called EvoGPT that integrates LLM-based test generation with evolutionary search techniques to create diverse, fault-revealing unit tests.
  arXiv  Detail & Related papers  (2025-05-18T13:48:53Z)
• On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementations [53.0667196725616]
  Deep Reinforcement Learning (DRL) is a paradigm of artificial intelligence where an agent uses a neural network to learn which actions to take in a given environment. DRL has recently gained traction from being able to solve complex environments like driving simulators, 3D robotic control, and multiplayer-online-battle-arena video games. Numerous implementations of the state-of-the-art algorithms responsible for training these agents, like the Deep Q-Network (DQN) and Proximal Policy Optimization (PPO) algorithms, currently exist.
  arXiv  Detail & Related papers  (2025-03-28T16:25:06Z)
• Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot [44.99833362998488]
  GenAI can be applied across numerous fields, with particular relevance in cybersecurity.<n>In this paper, we have analyzed the potential of leading generic-purpose GenAI tools.<n>Claude Opus, GPT-4 from ChatGPT, and Copilot-in augmenting the penetration testing process as defined by the Penetration Testing Execution Standard (PTES)
  arXiv  Detail & Related papers  (2025-01-12T22:48:37Z)
• Disrupting Test Development with AI Assistants [1.024113475677323]
  Generative AI-assisted coding tools like GitHub Copilot, ChatGPT, and Tabnine have significantly transformed software development. This paper analyzes how these innovations impact productivity and software test development metrics.
  arXiv  Detail & Related papers  (2024-11-04T17:52:40Z)
• ASTER: Natural and Multi-language Unit Test Generation with LLMs [6.259245181881262]
  We describe a generic pipeline that incorporates static analysis to guide LLMs in generating compilable and high-coverage test cases.<n>We conduct an empirical study to assess the quality of the generated tests in terms of code coverage and test naturalness.
  arXiv  Detail & Related papers  (2024-09-04T21:46:18Z)
• Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests [4.574205608859157]
  We introduce UTGen, which combines search-based software testing and large language models to enhance the understandability of automatically generated test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases.
  arXiv  Detail & Related papers  (2024-08-21T15:35:34Z)
• A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites [1.4563527353943984]
  Large Language Models (LLMs) have been applied to various aspects of software development. We present AgoneTest: an automated system for generating test suites for Java projects.
  arXiv  Detail & Related papers  (2024-08-14T23:02:16Z)
• Towards Automatic Generation of Amplified Regression Test Oracles [44.45138073080198]
  We propose a test oracle derivation approach to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour.
  arXiv  Detail & Related papers  (2023-07-28T12:38:44Z)
• CodeT: Code Generation with Generated Tests [49.622590050797236]
  We explore the use of pre-trained language models to automatically generate test cases. CodeT executes the code solutions using the generated test cases, and then chooses the best solution. We evaluate CodeT on five different pre-trained models with both HumanEval and MBPP benchmarks.
  arXiv  Detail & Related papers  (2022-07-21T10:18:37Z)
• AEON: A Method for Automatic Evaluation of NLP Test Cases [37.71980769922552]
  We use AEON to evaluate test cases generated by four popular testing techniques on five datasets across three typical NLP tasks. AEON achieves the best average precision in detecting semantic inconsistent test cases, outperforming the best baseline metric by 10%. AEON also has the highest average precision of finding unnatural test cases, surpassing the baselines by more than 15%.
  arXiv  Detail & Related papers  (2022-05-13T03:47:13Z)
• Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification [59.698811329287174]
  We leverage GPT-2 for generating artificial training instances in order to improve classification performance. Our results show that fine-tuning GPT-2 in a handful of label instances leads to consistent classification improvements.
  arXiv  Detail & Related papers  (2021-11-17T12:10:03Z)
• Online GANs for Automatic Performance Testing [0.10312968200748115]
  We present a novel algorithm for automatic performance testing that uses an online variant of the Generative Adversarial Network (GAN) The proposed approach does not require a prior training set or model of the system under test. We consider that the presented algorithm serves as a proof of concept and we hope that it can spark a research discussion on the application of GANs to test generation.
  arXiv  Detail & Related papers  (2021-04-21T06:03:27Z)
• Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework [68.96770035057716]
  A/B testing is a business strategy to compare a new product with an old one in pharmaceutical, technological, and traditional industries. This paper introduces a reinforcement learning framework for carrying A/B testing in online experiments.
  arXiv  Detail & Related papers  (2020-02-05T10:25:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.