Combining Type Inference and Automated Unit Test Generation for Python

• URL: http://arxiv.org/abs/2507.01477v1
• Date: Wed, 02 Jul 2025 08:41:28 GMT
• Title: Combining Type Inference and Automated Unit Test Generation for Python
• Authors: Lukas Krodinger, Stephan Lukasczyk, Gordon Fraser,
• Abstract summary: The lack of type information in dynamically-typed programming languages, such as Python, inhibits test generators.<n>We introduce type tracing, which extracts type-related information during execution and gradually refines the available type information.<n>The approach leads to up to 90.0% more branch coverage, improved mutation scores, and to type information of similar quality to that produced by other state-of-the-art type-inference tools.
• Score: 9.856068089918555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated unit test generation is an established research field that has so far focused on statically-typed programming languages. The lack of type information in dynamically-typed programming languages, such as Python, inhibits test generators, which heavily rely on information about parameter and return types of functions to select suitable arguments when constructing test cases. Since automated test generators inherently rely on frequent execution of candidate tests, we make use of these frequent executions to address this problem by introducing type tracing, which extracts type-related information during execution and gradually refines the available type information. We implement type tracing as an extension of the Pynguin test-generation framework for Python, allowing it (i) to infer parameter types by observing how parameters are used during runtime, (ii) to record the types of values that function calls return, and (iii) to use this type information to increase code coverage. The approach leads to up to 90.0% more branch coverage, improved mutation scores, and to type information of similar quality to that produced by other state-of-the-art type-inference tools.

Related papers

• LLM-based Unit Test Generation for Dynamically-Typed Programs [16.38145000434927]
  TypeTest is a novel framework that enhances type correctness in test generation through a vector-based Retrieval-Augmented Generation system.<n>In an evaluation on 125 real-world Python modules, TypeTest achieved an average statement coverage of 86.6% and branch coverage of 76.8%, outperforming state-of-theart tools by 5.4% and 9.3%, respectively.
  arXiv  Detail & Related papers  (2025-03-18T08:07:17Z)
• Toward a Corpus Study of the Dynamic Gradual Type [0.0]
  This paper reports on an in-progress corpus study of the dynamic type in Python, targeting 221 GitHub projects that use the mypy type checker.<n>The study reveals eight patterns-of-use for the dynamic type, which have implications for future refinements of the mypy type system and for tool support to encourage precise type annotations.
  arXiv  Detail & Related papers  (2025-03-11T22:18:51Z)
• Generative Type Inference for Python [62.01560866916557]
  This paper introduces TypeGen, a few-shot generative type inference approach that incorporates static domain knowledge from static analysis. TypeGen creates chain-of-thought (COT) prompts by translating the type inference steps of static analysis into prompts based on the type dependency graphs (TDGs) Experiments show that TypeGen outperforms the best baseline Type4Py by 10.0% for argument type prediction and 22.5% in return value type prediction in terms of top-1 Exact Match.
  arXiv  Detail & Related papers  (2023-07-18T11:40:31Z)
• Type Prediction With Program Decomposition and Fill-in-the-Type Training [2.7998963147546143]
  We build OpenTau, a search-based approach for type prediction that leverages large language models. We evaluate our work with a new dataset for TypeScript type prediction, and show that 47.4% of files type check (14.5% absolute improvement) with an overall rate of 3.3 type errors per file.
  arXiv  Detail & Related papers  (2023-05-25T21:16:09Z)
• TypeT5: Seq2seq Type Inference using Static Analysis [51.153089609654174]
  We present a new type inference method that treats type prediction as a code infilling task. Our method uses static analysis to construct dynamic contexts for each code element whose type signature is to be predicted by the model. We also propose an iterative decoding scheme that incorporates previous type predictions in the model's input context.
  arXiv  Detail & Related papers  (2023-03-16T23:48:00Z)
• Cross-Domain Evaluation of a Deep Learning-Based Type Inference System [0.44098366957385177]
  We investigate Type4Py as a representative of state-of-the-art deep learning-based type inference systems. We address the following problems: class imbalances, out-of-vocabulary words, dataset shifts, and unknown classes. Our dataset enables the evaluation of type inference systems in different domains of software projects.
  arXiv  Detail & Related papers  (2022-08-19T07:28:31Z)
• Query and Extract: Refining Event Extraction as Type-oriented Binary Decoding [51.57864297948228]
  We propose a novel event extraction framework that takes event types and argument roles as natural language queries. Our framework benefits from the attention mechanisms to better capture the semantic correlation between the event types or argument roles and the input text.
  arXiv  Detail & Related papers  (2021-10-14T15:49:40Z)
• Learning Sparse Prototypes for Text Generation [120.38555855991562]
  Prototype-driven text generation is inefficient at test time as a result of needing to store and index the entire training corpus. We propose a novel generative model that automatically learns a sparse prototype support set that achieves strong language modeling performance. In experiments, our model outperforms previous prototype-driven language models while achieving up to a 1000x memory reduction.
  arXiv  Detail & Related papers  (2020-06-29T19:41:26Z)
• LambdaNet: Probabilistic Type Inference using Graph Neural Networks [46.66093127573704]
  This paper proposes a probabilistic type inference scheme for TypeScript based on a graph neural network. Our approach can predict both standard types, like number or string, as well as user-defined types that have not been encountered during training.
  arXiv  Detail & Related papers  (2020-04-29T17:48:40Z)
• Multi-layer Optimizations for End-to-End Data Analytics [71.05611866288196]
  We introduce Iterative Functional Aggregate Queries (IFAQ), a framework that realizes an alternative approach. IFAQ treats the feature extraction query and the learning task as one program given in the IFAQ's domain-specific language. We show that a Scala implementation of IFAQ can outperform mlpack, Scikit, and specialization by several orders of magnitude for linear regression and regression tree models over several relational datasets.
  arXiv  Detail & Related papers  (2020-01-10T16:14:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.