TestMiner: Software Testing Analysis for GitHub Repositories

• URL: http://arxiv.org/abs/2607.12223v1
• Date: Mon, 13 Jul 2026 23:52:13 GMT
• Title: TestMiner: Software Testing Analysis for GitHub Repositories
• Abstract summary: TestMiner is a tool for exploring software testing in GitHub repositories.<n>TestMiner provides an overview of a project's testing practices, including test statistics, test location, test metrics across releases, and dependencies related to testing.
• Score: 2.1021180782525097
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software systems have unique testing characteristics. Some projects can emphasize unit tests, while others may focus on end-to-end testing. Test organization may vary across ecosystems: in languages like Python and Java, tests are typically placed in dedicated folders, whereas Go and Rust projects commonly co-locate tests with source code. These distinctions make it harder to understand how a project approaches testing. In this paper, we present TestMiner, a tool for exploring software testing in GitHub repositories. TestMiner provides an overview of a project's testing practices, including test statistics, test location, test metrics across releases, and dependencies related to testing. We used TestMiner in an undergraduate Software Testing course, where 50 students explored the testing practices of real-world GitHub repositories. Overall, students expressed positive feedback regarding TestMiner. They were able to critically explore a variety of testing practices, including test organization, test evolution, test fixtures, mocking, and edge-case testing. TestMiner is available at: https://andrehora.github.io/testminer. Screencast: https://youtu.be/w1sBgLTq-7Y.

Related papers

• WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing [57.7131457251794]
  We introduce WebTestBench, a benchmark for evaluating end-to-end automated web testing.<n>We decompose the testing process into two cascaded sub-tasks, checklist generation and defect detection, and propose WebTester.<n>Our findings expose a substantial gap between current computer-use agent capabilities and industrial-grade deployment demands.
  arXiv  Detail & Related papers  (2026-03-26T09:27:29Z)
• What Do Contribution Guidelines Say About Software Testing? [0.0]
  We analyze the guidelines of 200 Python and JavaScript open-source software projects.<n>We find that 78% of the projects include some form of test documentation for contributors.<n>Test documentation commonly explains how to run tests, but less often provides guidance on how to write tests.
  arXiv  Detail & Related papers  (2026-02-03T01:05:15Z)
• LLMs for Automated Unit Test Generation and Assessment in Java: The AgoneTest Framework [2.501198441875755]
  AgoneTest is an evaluation framework for Large Language Model-generated unit tests in Java.<n>For the subset of tests that compile, LLM-generated tests can match or exceed human-written tests in terms of coverage and defect detection.
  arXiv  Detail & Related papers  (2025-11-25T15:33:00Z)
• ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases [58.411135609139855]
  "Shortcuts" to complete tasks pose significant risks for reliable assessment and deployment of large language models.<n>We introduce ImpossibleBench, a benchmark framework that measures LLM agents' propensity to exploit test cases.<n>As a practical framework, ImpossibleBench is not just an evaluation but a versatile tool.
  arXiv  Detail & Related papers  (2025-10-23T06:58:32Z)
• Intention-Driven Generation of Project-Specific Test Cases [45.2380093475221]
  We propose IntentionTest, which generates project-specific tests given the description of validation intention.<n>We extensively evaluate IntentionTest against state-of-the-art baselines (DA, ChatTester, and EvoSuite) on 4,146 test cases from 13 open-source projects.
  arXiv  Detail & Related papers  (2025-07-28T08:35:04Z)
• ProjectTest: A Project-level LLM Unit Test Generation Benchmark and Impact of Error Fixing Mechanisms [48.43237545197775]
  Unit test generation has become a promising and important use case of LLMs.<n>ProjectTest is a project-level benchmark for unit test generation covering Python, Java, and JavaScript.
  arXiv  Detail & Related papers  (2025-02-10T15:24:30Z)
• Detecting and Evaluating Order-Dependent Flaky Tests in JavaScript [3.6513675781808357]
  Flaky tests pose a significant issue for software testing.<n>Previous research has identified test order dependency as one of the most prevalent causes of flakiness.<n>This paper aims to investigate test order dependency in JavaScript tests.
  arXiv  Detail & Related papers  (2025-01-22T06:52:11Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch.<n>Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests.<n> Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• TestSpark: IntelliJ IDEA's Ultimate Test Generation Companion [15.13443954421825]
  This paper introduces TestSpark, a plugin for IntelliJ IDEA that enables users to generate unit tests with only a few clicks. TestSpark also allows users to easily modify and run each generated test and integrate them into the project workflow.
  arXiv  Detail & Related papers  (2024-01-12T13:53:57Z)
• Do Automatic Test Generation Tools Generate Flaky Tests? [12.813573907094074]
  The prevalence and nature of flaky tests produced by test generation tools remain largely unknown. We generate tests using EvoSuite (Java) and Pynguin (Python) and execute each test 200 times. Our results show that flakiness is at least as common in generated tests as in developer-written tests.
  arXiv  Detail & Related papers  (2023-10-08T16:44:27Z)
• An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation [3.9762912548964864]
  This paper presents a large-scale empirical evaluation on the effectiveness of Large Language Models for automated unit test generation. We implement our approach in TestPilot, a test generation tool for JavaScript that automatically generates unit tests for all API functions in an npm package. We find that 92.8% of TestPilot's generated tests have no more than 50% similarity with existing tests.
  arXiv  Detail & Related papers  (2023-02-13T17:13:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.