Using Fourier Analysis and Mutant Clustering to Accelerate DNN Mutation Testing

• URL: http://arxiv.org/abs/2510.02718v1
• Date: Fri, 03 Oct 2025 04:36:42 GMT
• Title: Using Fourier Analysis and Mutant Clustering to Accelerate DNN Mutation Testing
• Authors: Ali Ghanbari, Sasan Tavakkol,
• Abstract summary: Deep neural network (DNN) mutation analysis is a promising approach to evaluating test set adequacy.<n>We present a technique, named DM#, for accelerating mutation testing using Fourier analysis.<n>Our results provide empirical evidence on the effectiveness of DM# in accelerating mutation testing by 28.38%, on average, at the average cost of only 0.72% error in mutation score.
• Score: 0.9617606953987995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural network (DNN) mutation analysis is a promising approach to evaluating test set adequacy. Due to the large number of generated mutants that must be tested on large datasets, mutation analysis is costly. In this paper, we present a technique, named DM#, for accelerating DNN mutation testing using Fourier analysis. The key insight is that DNN outputs are real-valued functions suitable for Fourier analysis that can be leveraged to quantify mutant behavior using only a few data points. DM# uses the quantified mutant behavior to cluster the mutants so that the ones with similar behavior fall into the same group. A representative from each group is then selected for testing, and the result of the test, e.g., whether the mutant is killed or survived, is reused for all other mutants represented by the selected mutant, obviating the need for testing other mutants. 14 DNN models of sizes ranging from thousands to millions of parameters, trained on different datasets, are used to evaluate DM# and compare it to several baseline techniques. Our results provide empirical evidence on the effectiveness of DM# in accelerating mutation testing by 28.38%, on average, at the average cost of only 0.72% error in mutation score. Moreover, on average, DM# incurs 11.78, 15.16, and 114.36 times less mutation score error compared to random mutant selection, boundary sample selection, and random sample selection techniques, respectively, while generally offering comparable speed-up.

Related papers

• Latent Space Class Dispersion: Effective Test Data Quality Assessment for DNNs [45.129846925131055]
  Latent Space Class Dispersion (LSCD) is a novel metric to quantify the quality of test datasets for Deep Neural Networks (DNNs)<n>Our empirical study shows that LSCD reveals and quantifies deficiencies in the test dataset of three popular benchmarks pertaining to image classification tasks.
  arXiv  Detail & Related papers  (2025-03-24T15:45:50Z)
• On Accelerating Deep Neural Network Mutation Analysis by Neuron and Mutant Clustering [1.7188280334580197]
  Mutation analysis of deep neural networks (DNNs) is a promising method for effective evaluation of test data quality and model robustness.<n>We present DEEPMAACC, a technique and a tool that speeds up mutation analysis through neuron and mutant clustering.<n>Our results demonstrate that a trade-off can be made between mutation testing speed and mutation score error.
  arXiv  Detail & Related papers  (2025-01-22T02:48:07Z)
• METFORD -- Mutation tEsTing Framework fOR anDroid [0.0]
  This research aims to contribute to reducing Android mutation testing costs.<n>It implements mutation testing operators according to mutant schemata.<n>Additional mutation operators can be implemented in JavaScript and easily integrated into the framework.
  arXiv  Detail & Related papers  (2025-01-06T09:36:57Z)
• Latent Mutants: A large-scale study on the Interplay between mutation testing and software evolution [2.1984302611206537]
  We study the characteristics of what we call latent mutants, i.e., the mutants that are live in one version and killed in later revisions.<n>We examine 131,308 mutants generated by Pitest on 13 open-source projects.
  arXiv  Detail & Related papers  (2025-01-03T15:44:38Z)
• An Empirical Evaluation of Manually Created Equivalent Mutants [54.02049952279685]
  Less than 10 % of manually created mutants are equivalent. Surprisingly, our findings indicate that a significant portion of developers struggle to accurately identify equivalent mutants.
  arXiv  Detail & Related papers  (2024-04-14T13:04:10Z)
• Predicting loss-of-function impact of genetic mutations: a machine learning approach [0.0]
  This paper aims to train machine learning models on the attributes of a genetic mutation to predict LoFtool scores. These attributes included, but were not limited to, the position of a mutation on a chromosome, changes in amino acids, and changes in codons caused by the mutation. Models were evaluated using five-fold cross-validated averages of r-squared, mean squared error, root mean squared error, mean absolute error, and explained variance.
  arXiv  Detail & Related papers  (2024-01-26T19:27:38Z)
• MuRS: Mutant Ranking and Suppression using Identifier Templates [4.9205581820379765]
  Google's mutation testing service integrates diff-based mutation testing into the code review process. Google's mutation testing service implements a number of suppression rules, which target not-useful mutants. This paper proposes and evaluates MuRS, an automated approach that groups mutants by patterns in the source code under test.
  arXiv  Detail & Related papers  (2023-06-15T13:43:52Z)
• Equivariance Allows Handling Multiple Nuisance Variables When Analyzing Pooled Neuroimaging Datasets [53.34152466646884]
  In this paper, we show how bringing recent results on equivariant representation learning instantiated on structured spaces together with simple use of classical results on causal inference provides an effective practical solution. We demonstrate how our model allows dealing with more than one nuisance variable under some assumptions and can enable analysis of pooled scientific datasets in scenarios that would otherwise entail removing a large portion of the samples.
  arXiv  Detail & Related papers  (2022-03-29T04:54:06Z)
• StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder [48.2010192865749]
  Unsupervised anomaly detection (UAD) can learn a data distribution from an unlabelled dataset of healthy subjects and then be applied to detect out of distribution samples. This research proposes a compact version of the "context-encoding" VAE (ceVAE) model, combined with pre and post-processing steps, creating a UAD pipeline (StRegA) The proposed pipeline achieved a Dice score of 0.642$pm$0.101 while detecting tumours in T2w images of the BraTS dataset and 0.859$pm$0.112 while detecting artificially induced anomalies.
  arXiv  Detail & Related papers  (2022-01-31T14:27:35Z)
• Debiased Graph Neural Networks with Agnostic Label Selection Bias [59.61301255860836]
  Most existing Graph Neural Networks (GNNs) are proposed without considering the selection bias in data. We propose a novel Debiased Graph Neural Networks (DGNN) with a differentiated decorrelation regularizer. Our proposed model outperforms the state-of-the-art methods and DGNN is a flexible framework to enhance existing GNNs.
  arXiv  Detail & Related papers  (2022-01-19T16:50:29Z)
• Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation [154.2195491708548]
  We study the prediction discriminability and diversity by studying the structure of the classification output matrix of a randomly selected data batch. We propose Batch Nuclear-norm Maximization and Minimization, which performs nuclear-norm on the target output matrix to enhance the target prediction ability. Experiments show that our method could boost the adaptation accuracy and robustness under three typical domain adaptation scenarios.
  arXiv  Detail & Related papers  (2021-07-13T15:08:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.