Exploring the Potential of Large Language Models in Simulink-Stateflow Mutant Generation

• URL: http://arxiv.org/abs/2602.04066v1
• Date: Tue, 03 Feb 2026 23:01:47 GMT
• Title: Exploring the Potential of Large Language Models in Simulink-Stateflow Mutant Generation
• Abstract summary: Large Language Models (LLMs) can generate high-quality, domain-specific mutants for Simulink-Stateflow models.<n>LLMs generate mutants up to 13x faster than a manually engineered mutation-based baseline.<n>Our analysis reveals that few-shot prompting combined with low-to-medium temperature values yields optimal results.
• Score: 11.706481833926818
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mutation analysis is a powerful technique for assessing test-suite adequacy, yet conventional approaches suffer from generating redundant, equivalent, or non-executable mutants. These challenges are particularly amplified in Simulink-Stateflow models due to the hierarchical structure these models have, which integrate continuous dynamics with discrete-event behaviors and are widely deployed in safety-critical Cyber-Physical Systems (CPSs). While prior work has explored machine learning and manually engineered mutation operators, these approaches remain constrained by limited training data and scalability issues. Motivated by recent advances in Large Language Models (LLMs), we investigate their potential to generate high-quality, domain-specific mutants for Simulink-Stateflow models. We develop an automated pipeline that converts Simulink-Stateflow models to structured JSON representations and systematically evaluates different mutation and prompting strategies across eight state-of-the-art LLMs. Through a comprehensive empirical study involving 38,400 LLM-generated mutants across four Simulink-Stateflow models, we demonstrate that LLMs generate mutants up to 13x faster than a manually engineered mutation-based baseline while producing significantly fewer equivalent and duplicate mutants and consistently achieving superior mutant quality. Moreover, our analysis reveals that few-shot prompting combined with low-to-medium temperature values yields optimal results. We provide an open-source prototype tool and release our complete dataset to facilitate reproducibility and advance future research in this domain.

Related papers

• Hype Meets Reality: Large Language Models as Mutators in Search-based Automated Program Repair of Simulink-Stateflow Models [9.216325369400606]
  We extend the state-of-the-art FlowRepair approach by replacing a subset of its mutation operators with Large Language Models (LLMs)<n>Contrary to expectations, the LLM-based mutation substantially degrades repair performance under the FlowRepair experimental setup.<n>Our analysis reveals that, in this integration, LLMs struggle with precise symbolic edits, lack behavioral feedback, and generate a noisy search space.
  arXiv  Detail & Related papers  (2026-08-19T18:09:54Z)
• Model-Driven Discipline for Multi-Agent LLMs: Requirement-to-Verification Generation of Traceable System Models [62.62160094849002]
  RADIANT is an engineering methodology that combines MDE with Multi-Agent Large Language Models.<n>It automatically generates heterogeneous models across engineering phases.<n>It provides exact, automated change-impact analysis.
  arXiv  Detail & Related papers  (2026-07-18T08:50:55Z)
• Neural Controlled Differential Equations for EMT-Level Surrogate Modeling of Grid-Forming Inverters [13.252561724661085]
  This paper proposes a Neural Controlled Differential Equation framework for learning continuous-time surrogate models of grid-forming inverters for electromagnetic transient (EMT) simulation.<n>A physics-inspired regularization method is utilized to enhance stability and coherence.<n>The results show that Neural CDE-based component modeling offers a physically consistent surrogate modeling approach for EMT-level simulation studies.
  arXiv  Detail & Related papers  (2026-06-28T10:47:26Z)
• Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations [1.58310730488265]
  In industrial domains, Functional Mock-up Interface (FMI) is used to exchange simulation models as Functional Mock-up Units (FMUs)<n> deriving effective test oracles for these simulation models remains challenging due to the absence of explicit expected outputs.<n>We propose an LLM-powered multi-agent workflow for specification-based metamorphic testing of FMU-based simulation models.
  arXiv  Detail & Related papers  (2026-05-24T14:30:56Z)
• A Flow Model with Low-Rank Transformers for Incomplete Multimodal Survival Analysis [36.102030480314816]
  We propose a novel framework that combines a low-rank Transformer with a flow-based generative model for robust and flexible multimodal survival prediction.<n>Our method not only achieves state-of-the-art performance under complete modality settings, but also maintains robust and superior accuracy under the incomplete modalities scenario.
  arXiv  Detail & Related papers  (2025-10-22T02:27:05Z)
• Dynamicasome: a molecular dynamics-guided and AI-driven pathogenicity prediction catalogue for all genetic mutations [1.5071448753819772]
  We show that integrating detailed conformational data extracted from molecular dynamics simulations into advanced AI-based models increases their predictive power.<n>We carry out an exhaustive mutational analysis of the disease gene PMM2 and subject structural models of each variant to MDS.<n>Our best performing model, a neuronal networks model, also predicts the pathogenicity of several PMM2 mutations currently considered of unknown signi cance.
  arXiv  Detail & Related papers  (2025-09-23T17:33:05Z)
• Transition Models: Rethinking the Generative Learning Objective [68.16330673177207]
  We introduce a continuous-time dynamics equation that analytically defines state transitions across any finite time interval.<n>This leads to a novel generative paradigm, Transition Models (TiM), which adapt to arbitrary-step transitions.<n>TiM achieves state-of-the-art performance, surpassing leading models such as SD3.5 (8B parameters) and FLUX.1 (12B parameters) across all evaluated step counts.
  arXiv  Detail & Related papers  (2025-09-04T17:05:59Z)
• AI enhanced data assimilation and uncertainty quantification applied to Geological Carbon Storage [0.0]
  We introduce the Surrogate-based hybrid ESMDA (SH-ESMDA), an adaptation of the traditional Ensemble Smoother with Multiple Data Assimilation (ESMDA) We also introduce Surrogate-based Hybrid RML (SH-RML), a variational data assimilation approach that relies on the randomized maximum likelihood (RML) Our comparative analyses show that SH-RML offers better uncertainty compared to conventional ESMDA for the case study.
  arXiv  Detail & Related papers  (2024-02-09T00:24:46Z)
• Benchmarking Machine Learning Robustness in Covid-19 Genome Sequence Classification [109.81283748940696]
  We introduce several ways to perturb SARS-CoV-2 genome sequences to mimic the error profiles of common sequencing platforms such as Illumina and PacBio. We show that some simulation-based approaches are more robust (and accurate) than others for specific embedding methods to certain adversarial attacks to the input sequences.
  arXiv  Detail & Related papers  (2022-07-18T19:16:56Z)
• Multi-fidelity Hierarchical Neural Processes [79.0284780825048]
  Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling. We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2022-06-10T04:54:13Z)
• Bayesian Active Learning for Discrete Latent Variable Models [19.852463786440122]
  Active learning seeks to reduce the amount of data required to fit the parameters of a model. latent variable models play a vital role in neuroscience, psychology, and a variety of other engineering and scientific disciplines.
  arXiv  Detail & Related papers  (2022-02-27T19:07:12Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Exploring the potential of transfer learning for metamodels of heterogeneous material deformation [0.0]
  We show that transfer learning can be used to leverage both low-fidelity simulation data and simulation data. We extend Mechanical MNIST, our open source benchmark dataset of heterogeneous material undergoing large deformation. We show that transferring the knowledge stored in metamodels trained on these low-fidelity simulation results can vastly improve the performance of metamodels used to predict the results of high-fidelity simulations.
  arXiv  Detail & Related papers  (2020-10-28T12:43:46Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.