From Requirements to Test Cases: An NLP-Based Approach for High-Performance ECU Test Case Automation

• URL: http://arxiv.org/abs/2505.00547v1
• Date: Thu, 01 May 2025 14:23:55 GMT
• Title: From Requirements to Test Cases: An NLP-Based Approach for High-Performance ECU Test Case Automation
• Authors: Nikitha Medeshetty, Ahmad Nauman Ghazi, Sadi Alawadi, Fahed Alkhabbas,
• Abstract summary: This study investigates the use of Natural Language Processing techniques to transform natural language requirements into structured test case specifications.<n>A dataset of 400 feature element documents was used to evaluate both approaches for extracting key elements such as signal names and values.<n>The Rule-Based method outperforms the NER method, achieving 95% accuracy for more straightforward requirements with single signals.
• Score: 0.5249805590164901
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automating test case specification generation is vital for improving the efficiency and accuracy of software testing, particularly in complex systems like high-performance Electronic Control Units (ECUs). This study investigates the use of Natural Language Processing (NLP) techniques, including Rule-Based Information Extraction and Named Entity Recognition (NER), to transform natural language requirements into structured test case specifications. A dataset of 400 feature element documents from the Polarion tool was used to evaluate both approaches for extracting key elements such as signal names and values. The results reveal that the Rule-Based method outperforms the NER method, achieving 95% accuracy for more straightforward requirements with single signals, while the NER method, leveraging SVM and other machine learning algorithms, achieved 77.3% accuracy but struggled with complex scenarios. Statistical analysis confirmed that the Rule-Based approach significantly enhances efficiency and accuracy compared to manual methods. This research highlights the potential of NLP-driven automation in improving quality assurance, reducing manual effort, and expediting test case generation, with future work focused on refining NER and hybrid models to handle greater complexity.

Related papers

• Mixed-Precision Conjugate Gradient Solvers with RL-Driven Precision Tuning [0.0]
  This paper presents a novel reinforcement learning (RL) framework for dynamically optimizing numerical precision.<n>We employ Q-learning to adaptively assign precision levels to key operations, striking an optimal balance between computational efficiency and numerical accuracy.<n>Results demonstrate the effectiveness of RL in enhancing solver's performance, marking the first application of RL to mixed-precision numerical methods.
  arXiv  Detail & Related papers  (2025-04-19T11:35:03Z)
• TMIQ: Quantifying Test and Measurement Domain Intelligence in Large Language Models [0.0]
  We introduce the Test and Measurement Intelligence Quotient (TMIQ), a benchmark designed to quantitatively assess Large Language Models (LLMs)<n>TMIQ offers a comprehensive set of scenarios and metrics for detailed evaluation, including SCPI command matching accuracy, ranked response evaluation, Chain-of-Thought Reasoning (CoT)<n>In testing various LLMs, our findings indicate varying levels of proficiency, with exact SCPI command match accuracy ranging from around 56% to 73%, and ranked matching first-position scores achieving around 33%.
  arXiv  Detail & Related papers  (2025-03-03T23:12:49Z)
• AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models [86.83875864328984]
  We propose an automated method for synthesizing open-ended logic puzzles, and use it to develop a bilingual benchmark, AutoLogi.<n>Our approach features program-based verification and controllable difficulty levels, enabling more reliable evaluation that better distinguishes models' reasoning abilities.
  arXiv  Detail & Related papers  (2025-02-24T07:02:31Z)
• Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
  We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios. We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-06-07T11:37:45Z)
• Enriching Automatic Test Case Generation by Extracting Relevant Test Inputs from Bug Reports [10.587260348588064]
  We introduce BRMiner, a novel approach that leverages Large Language Models (LLMs) in combination with traditional techniques to extract relevant inputs from bug reports.<n>In this study, we evaluate BRMiner using the Defects4J benchmark and test generation tools such as EvoSuite and Randoop.<n>Our results demonstrate that BRMiner achieves a Relevant Input Rate (RIR) of 60.03% and a Relevant Input Extraction Accuracy Rate (RIEAR) of 31.71%.
  arXiv  Detail & Related papers  (2023-12-22T18:19:33Z)
• Efficient Learning of Accurate Surrogates for Simulations of Complex Systems [0.0]
  We introduce an online learning method empowered by sampling-driven sampling. It ensures that all turning points on the model response surface are included in the training data. We apply our method to simulations of nuclear matter to demonstrate that highly accurate surrogates can be reliably auto-generated.
  arXiv  Detail & Related papers  (2022-07-11T20:51:11Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks [88.62288327934499]
  We propose a novel augmentation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-11-03T07:49:15Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Machine Learning to Tackle the Challenges of Transient and Soft Errors in Complex Circuits [0.16311150636417257]
  Machine learning models are used to predict accurate per-instance Functional De-Rating data for the full list of circuit instances. The presented methodology is applied on a practical example and various machine learning models are evaluated and compared.
  arXiv  Detail & Related papers  (2020-02-18T18:38:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.