Automated identification of metamorphic test scenarios for an ocean-modeling application

• URL: http://arxiv.org/abs/2009.01554v1
• Date: Thu, 3 Sep 2020 10:03:56 GMT
• Title: Automated identification of metamorphic test scenarios for an ocean-modeling application
• Authors: Dilip J. Hiremath, Martin Claus, Wilhelm Hasselbring, Willi Rath
• Abstract summary: We present work in progress for automated generation of metamorphic test scenarios using machine learning. Our application domain is ocean modeling, where test oracles often do not exist, but where symmetries of the simulated physical systems are known.
• Score: 0.802904964931021
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Metamorphic testing seeks to validate software in the absence of test oracles. Our application domain is ocean modeling, where test oracles often do not exist, but where symmetries of the simulated physical systems are known. In this short paper we present work in progress for automated generation of metamorphic test scenarios using machine learning. Metamorphic testing may be expressed as f(g(X))=h(f(X)) with f being the application under test, with input data X, and with the metamorphic relation (g, h). Automatically generated metamorphic relations can be used for constructing regression tests, and for comparing different versions of the same software application. Here, we restrict to h being the identity map. Then, the task of constructing tests means finding different g which we tackle using machine learning algorithms. These algorithms typically minimize a cost function. As one possible g is already known to be the identity map, for finding a second possible g, we construct the cost function to minimize for g being a metamorphic relation and to penalize for g being the identity map. After identifying the first metamorphic relation, the procedure is repeated with a cost function rewarding g that are orthogonal to previously found metamorphic relations. For experimental evaluation, two implementations of an ocean-modeling application will be subjected to the proposed method with the objective of presenting the use of metamorphic relations to test the implementations of the applications.

Related papers

• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Systematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing Perspective [1.6799377888527685]
  We propose three new classes of metamorphic relations, which address the properties of systematicity, compositionality and transitivity. With them, we test the internal consistency of state-of-the-art NLP models, and show that they do not always behave according to their expected linguistic properties. Lastly, we introduce a novel graphical notation that efficiently summarises the inner structure of metamorphic relations.
  arXiv  Detail & Related papers  (2022-04-26T13:50:07Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Using Metamorphic Relations to Verify and Enhance Artcode Classification [39.36253474867746]
  An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a set of predefined classes. An approach to software testing that alleviates the oracle problem is metamorphic testing (MT) This paper examines the problem of classifying images containing visually hidden markers called Artcodes, and applies MT to verify and enhance the trained classifiers.
  arXiv  Detail & Related papers  (2021-08-05T15:54:56Z)
• Finding Geometric Models by Clustering in the Consensus Space [61.65661010039768]
  We propose a new algorithm for finding an unknown number of geometric models, e.g., homographies. We present a number of applications where the use of multiple geometric models improves accuracy. These include pose estimation from multiple generalized homographies; trajectory estimation of fast-moving objects.
  arXiv  Detail & Related papers  (2021-03-25T14:35:07Z)
• Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing [0.39146761527401414]
  StackGAN-v2 has proven capable of generating high resolution images that reflect the details specified in the input text descriptions. We adopt Metamorphic Testing technique to evaluate the robustness of the model with a variety of unexpected training dataset. We find that StackGAN-v2 algorithm is susceptible to input images with obtrusive objects, even if it overlaps with the main object minimally.
  arXiv  Detail & Related papers  (2021-03-04T07:29:17Z)
• Model identification and local linear convergence of coordinate descent [74.87531444344381]
  We show that cyclic coordinate descent achieves model identification in finite time for a wide class of functions. We also prove explicit local linear convergence rates for coordinate descent.
  arXiv  Detail & Related papers  (2020-10-22T16:03:19Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)
• Query Training: Learning a Worse Model to Infer Better Marginals in Undirected Graphical Models with Hidden Variables [11.985433487639403]
  Probabilistic graphical models (PGMs) provide a compact representation of knowledge that can be queried in a flexible way. We introduce query training (QT), a mechanism to learn a PGM that is optimized for the approximate inference algorithm that will be paired with it. We demonstrate experimentally that QT can be used to learn a challenging 8-connected grid Markov random field with hidden variables.
  arXiv  Detail & Related papers  (2020-06-11T20:34:32Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)
• Object-based Metamorphic Testing through Image Structuring [0.6445605125467573]
  Testing software is often costly due to the need of mass-producing test cases and providing a test oracle for it. One method that has been proposed in order to alleviate the oracle problem is metamorphic testing.
  arXiv  Detail & Related papers  (2020-02-12T10:32:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.