Automated Statistical Testing and Certification of a Reliable Model-Coupling Server for Scientific Computing

• URL: http://arxiv.org/abs/2505.09769v2
• Date: Mon, 30 Jun 2025 15:20:34 GMT
• Title: Automated Statistical Testing and Certification of a Reliable Model-Coupling Server for Scientific Computing
• Abstract summary: Sequence-based specification and usage-driven statistical testing are designed for rigorous and cost-effective software development.<n>We focus on the quality assurance of the more elastic web service via a combination of rigorous specification and testing methods.
• Score: 0.7100520098029439
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Sequence-based specification and usage-driven statistical testing are designed for rigorous and cost-effective software development, offering a semi-formal approach to assessing the behavior of complex systems and interactions between various components. This approach is particularly valuable for scientific computing applications in which comprehensive tests are needed to prevent flawed results or conclusions. As scientific discovery becomes increasingly more complex, domain scientists couple multiple scientific computing models or simulations to solve intricate multiphysics and multiscale problems. These model-coupling applications use a hardwired coupling program or a flexible web service to link and combine different models. In this paper, we focus on the quality assurance of the more elastic web service via a combination of rigorous specification and testing methods. The application of statistical testing exposes problems ignored by pre-written unit tests and highlights areas in the code where failures might occur. We certify the model-coupling server controller with a derived reliability statistic, offering a quantitative measure to support a claim of its robustness.

Related papers

• A systematic literature Review for Transformer-based Software Vulnerability detection [0.8409304328108455]
  This SLR critically analysed 80 studies published between 2021 and 2025 that utilised transformer models to identify software vulnerabilities.<n>We classify transformer models into encoder, decoder, and combined architectures and analyse both pre-trained and fine-tuned versions utilized on source code, logs, and smart contracts.
  arXiv  Detail & Related papers  (2026-04-27T14:26:19Z)
• Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systems [54.916243942641444]
  Large language models (LLMs) are emerging as key enablers of automation in domains such as telecommunications.<n>We study an edge-cloud-expert cascaded LLM-based knowledge system that supports decision-making through a question-and-answer pipeline.
  arXiv  Detail & Related papers  (2025-12-23T03:10:09Z)
• An Agentic Framework for Autonomous Materials Computation [70.24472585135929]
  Large Language Models (LLMs) have emerged as powerful tools for accelerating scientific discovery.<n>Recent advances integrate LLMs into agentic frameworks, enabling retrieval, reasoning, and tool use for complex scientific experiments.<n>Here, we present a domain-specialized agent designed for reliable automation of first-principles materials computations.
  arXiv  Detail & Related papers  (2025-12-22T15:03:57Z)
• Using Causal Inference to Test Systems with Hidden and Interacting Variables: An Evaluative Case Study [2.1146241717926664]
  Software systems with large parameter spaces, nondeterminism and high computational cost are challenging to test.<n>Recent software testing techniques based on causal inference have been successfully applied to systems that exhibit such characteristics.
  arXiv  Detail & Related papers  (2025-04-23T08:50:24Z)
• A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection [55.2480439325792]
  This paper presents a hybrid framework that integrates both statistical feature selection and classification techniques to improve defect detection accuracy.<n>We present around 55 distinguished features that are extracted from industrial images, which are then analyzed using statistical methods.<n>By integrating these methods with flexible machine learning applications, the proposed framework improves detection accuracy and reduces false positives and misclassifications.
  arXiv  Detail & Related papers  (2024-12-11T22:12:21Z)
• AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions [45.0447118979891]
  AutoKaggle implements an iterative development process that combines code execution and unit testing to ensure code correctness and logic consistency. Our universal data science toolkit, comprising validated functions for data cleaning, feature engineering, and modeling, forms the foundation of this solution. AutoKaggle achieves a validation rate of 0.85 and a comprehensive score of 0.82 in typical data science pipelines.
  arXiv  Detail & Related papers  (2024-10-27T12:44:25Z)
• FedRFQ: Prototype-Based Federated Learning with Reduced Redundancy, Minimal Failure, and Enhanced Quality [41.88338945821504]
  FedRFQ is a prototype-based federated learning approach that aims to reduce redundancy, minimize failures, and improve underlinequality. We introduce the BFT-detect, a BFT (Byzantine Fault Tolerance) detectable aggregation algorithm, to ensure the security of FedRFQ against poisoning attacks and server malfunctions.
  arXiv  Detail & Related papers  (2024-01-15T09:50:27Z)
• mlscorecheck: Testing the consistency of reported performance scores and experiments in machine learning [0.0]
  We have developed numerical techniques capable of identifying inconsistencies between reported performance scores and various experimental setups in machine learning problems. These consistency tests are integrated into the open-source package mlscorecheck.
  arXiv  Detail & Related papers  (2023-11-13T18:31:48Z)
• Deep anytime-valid hypothesis testing [29.273915933729057]
  We propose a general framework for constructing powerful, sequential hypothesis tests for nonparametric testing problems. We develop a principled approach of leveraging the representation capability of machine learning models within the testing-by-betting framework. Empirical results on synthetic and real-world datasets demonstrate that tests instantiated using our general framework are competitive against specialized baselines.
  arXiv  Detail & Related papers  (2023-10-30T09:46:19Z)
• Constrained Adversarial Learning for Automated Software Testing: a literature review [0.0]
  This literature review is focused on the current state-of-the-art of constrained data generation approaches applied for adversarial learning and software testing.<n>The advantages and limitations of those specific for white-box, grey-box, and black-box testing were analyzed.
  arXiv  Detail & Related papers  (2023-03-14T00:27:33Z)
• Leveraging a Probabilistic PCA Model to Understand the Multivariate Statistical Network Monitoring Framework for Network Security Anomaly Detection [64.1680666036655]
  We revisit anomaly detection techniques based on PCA from a probabilistic generative model point of view. We have evaluated the mathematical model using two different datasets.
  arXiv  Detail & Related papers  (2023-02-02T13:41:18Z)
• Testing Causality in Scientific Modelling Software [0.26388783516590225]
  Causal Testing Framework is a framework that uses Causal Inference techniques to establish causal effects from existing data. We present three case studies covering real-world scientific models, demonstrating how the Causal Testing Framework can infer metamorphic test outcomes.
  arXiv  Detail & Related papers  (2022-09-01T10:57:54Z)
• Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control [67.52000805944924]
  Learn then Test (LTT) is a framework for calibrating machine learning models. Our main insight is to reframe the risk-control problem as multiple hypothesis testing. We use our framework to provide new calibration methods for several core machine learning tasks with detailed worked examples in computer vision.
  arXiv  Detail & Related papers  (2021-10-03T17:42:03Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)
• A New Score for Adaptive Tests in Bayesian and Credal Networks [64.80185026979883]
  A test is adaptive when its sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain.
  arXiv  Detail & Related papers  (2021-05-25T20:35:42Z)
• Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design [76.29738151117583]
  Federated Learning (FL) has become a promising tool for training effective machine learning models among distributed clients. However, low quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training. We model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk.
  arXiv  Detail & Related papers  (2021-05-10T08:02:27Z)
• Data Driven Testing of Cyber Physical Systems [12.93632948681342]
  We propose an approach to automatically generate fault-revealing test cases for CPS. Data collected from an application managing a smart building have been used to learn models of the environment.
  arXiv  Detail & Related papers  (2021-02-23T04:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.