Intent-Based Mutation Testing: From Naturally Written Programming Intents to Mutants

• URL: http://arxiv.org/abs/2607.05149v1
• Date: Mon, 06 Jul 2026 14:30:02 GMT
• Title: Intent-Based Mutation Testing: From Naturally Written Programming Intents to Mutants
• Abstract summary: intent-based mutation testing generates mutations by changing the programming intents that are implemented in the programs under test.<n>We implement intent-based mutation testing using Large Language Models (LLMs) that mutate programming intents and transform them into mutants.<n>We evaluate intent-based mutation on 29 programs and show that it generates mutations that are syntactically complex, semantically diverse, and quite different (semantically) from the traditional ones.
• Score: 5.317864735982288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents intent-based mutation testing, a testing approach that generates mutations by changing the programming intents that are implemented in the programs under test. In contrast to traditional mutation testing, which changes (mutates) the way programs are written, intent mutation changes (mutates) the behavior of the programs by producing mutations that implement (slightly) different intents than those implemented in the original program. The mutations of the programming intents represent possible corner cases and misunderstandings of the program behavior, i.e., program specifications, and thus can capture different classes of faults than traditional (syntax-based) mutation. Moreover, since programming intents can be implemented in different ways, intent-based mutation testing can generate diverse and complex mutations that are close to the original programming intents (specifications) and thus direct testing towards the intent variants of the program behavior/specifications. We implement intent-based mutation testing using Large Language Models (LLMs) that mutate programming intents and transform them into mutants. We evaluate intent-based mutation on 29 programs and show that it generates mutations that are syntactically complex, semantically diverse, and quite different (semantically) from the traditional ones. We also show that 55% of the intent-based mutations are not subsumed by traditional mutations. Overall, our analysis shows that intent-based mutation testing can be a powerful complement to traditional (syntax-based) mutation testing.

Related papers

• Round-Trip Mutation Testing: Translating Code to Natural Language Intent and back [5.4804081616439975]
  This paper presents Round-Trip Mutation Testing (RTM)<n>RTM generates mutants from LLM mistranslations between a program code and its intent.<n>Our results demonstrate the potential of round-trip mutation testing to produce syntactically more diverse mutants.
  arXiv  Detail & Related papers  (2026-07-03T11:35:58Z)
• Transducing Language Models [52.080921891265255]
  We introduce a framework for language models derived from deterministic string-to-string transformations.<n>We develop algorithms that compose a language model with an FST to *marginalize* over source strings mapping to a given target.<n>We present an exact algorithm, an efficient approximation, and a theoretical analysis.
  arXiv  Detail & Related papers  (2026-03-05T14:04:08Z)
• Probability Distributions Computed by Hard-Attention Transformers [53.17368795629463]
  We show that making transformer language recognizers autoregressive can sometimes increase their expressivity.<n>Our overall contribution is to tease apart what functions transformers are capable of expressing, in their most common use-case as language models.
  arXiv  Detail & Related papers  (2025-10-31T02:41:05Z)
• METFORD -- Mutation tEsTing Framework fOR anDroid [0.0]
  This research aims to contribute to reducing Android mutation testing costs.<n>It implements mutation testing operators according to mutant schemata.<n>Additional mutation operators can be implemented in JavaScript and easily integrated into the framework.
  arXiv  Detail & Related papers  (2025-01-06T09:36:57Z)
• MMT: Mutation Testing of Java Bytecode with Model Transformation -- An Illustrative Demonstration [0.11470070927586014]
  mutation testing is an approach to check the robustness of test suites. We propose a model-driven approach where mutations of Java bytecode can be flexibly defined by model transformation. The corresponding tool called MMT has been extended with advanced mutation operators for modifying object-oriented structures.
  arXiv  Detail & Related papers  (2024-04-22T11:33:21Z)
• LLMorpheus: Mutation Testing using Large Language Models [5.448283690603358]
  This paper presents a technique for mutation testing where placeholders are introduced at designated locations in a program's source code.<n>We find LLMorpheus to be capable of producing mutants that resemble existing bugs that cannot be produced by StrykerJS.
  arXiv  Detail & Related papers  (2024-04-15T17:25:14Z)
• An Empirical Evaluation of Manually Created Equivalent Mutants [54.02049952279685]
  Less than 10 % of manually created mutants are equivalent. Surprisingly, our findings indicate that a significant portion of developers struggle to accurately identify equivalent mutants.
  arXiv  Detail & Related papers  (2024-04-14T13:04:10Z)
• Learning Transformer Programs [78.9509560355733]
  We introduce a procedure for training Transformers that are mechanistically interpretable by design. Instead of compiling human-written programs into Transformers, we design a modified Transformer that can be trained using gradient-based optimization. The Transformer Programs can automatically find reasonable solutions, performing on par with standard Transformers of comparable size.
  arXiv  Detail & Related papers  (2023-06-01T20:27:01Z)
• Self-Supervised Learning for Group Equivariant Neural Networks [75.62232699377877]
  Group equivariant neural networks are the models whose structure is restricted to commute with the transformations on the input. We propose two concepts for self-supervised tasks: equivariant pretext labels and invariant contrastive loss. Experiments on standard image recognition benchmarks demonstrate that the equivariant neural networks exploit the proposed self-supervised tasks.
  arXiv  Detail & Related papers  (2023-03-08T08:11:26Z)
• Composed Variational Natural Language Generation for Few-shot Intents [118.37774762596123]
  We generate training examples for few-shot intents in the realistic imbalanced scenario. To evaluate the quality of the generated utterances, experiments are conducted on the generalized few-shot intent detection task. Our proposed model achieves state-of-the-art performances on two real-world intent detection datasets.
  arXiv  Detail & Related papers  (2020-09-21T17:48:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.