Object-based Metamorphic Testing through Image Structuring

• URL: http://arxiv.org/abs/2002.07046v1
• Date: Wed, 12 Feb 2020 10:32:18 GMT
• Title: Object-based Metamorphic Testing through Image Structuring
• Authors: Adrian Wildandyawan, Yasuharu Nishi
• Abstract summary: Testing software is often costly due to the need of mass-producing test cases and providing a test oracle for it. One method that has been proposed in order to alleviate the oracle problem is metamorphic testing.
• Score: 0.6445605125467573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Testing software is often costly due to the need of mass-producing test cases and providing a test oracle for it. This is often referred to as the oracle problem. One method that has been proposed in order to alleviate the oracle problem is metamorphic testing. Metamorphic testing produces new test cases by altering an existing test case, and uses the metamorphic relation between the inputs and the outputs of the System Under Test (SUT) to predict the expected outputs of the produced test cases. Metamorphic testing has often been used for image processing software, where changes are applied to the image's attributes to create new test cases with annotations that are the same as the original image. We refer to this existing method as the image-based metamorphic testing. In this research, we propose an object-based metamorphic testing and a composite metamorphic testing which combines different metamorphic testing approaches to relatively increase test coverage.

Related papers

• Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models [49.06068319380296]
  We introduce context-aware testing (CAT) which uses context as an inductive bias to guide the search for meaningful model failures. We instantiate the first CAT system, SMART Testing, which employs large language models to hypothesize relevant and likely failures.
  arXiv  Detail & Related papers  (2024-10-31T15:06:16Z)
• Few-Shot Anomaly Detection via Category-Agnostic Registration Learning [65.64252994254268]
  Most existing anomaly detection methods require a dedicated model for each category. This article proposes a novel few-shot AD (FSAD) framework. It is the first FSAD method that requires no model fine-tuning for novel categories.
  arXiv  Detail & Related papers  (2024-06-13T05:01:13Z)
• Security Testing of RESTful APIs With Test Case Mutation [0.0]
  This paper proposes an automated approach to help developers generate test cases for experimenting with each service in isolation. We present our test case mutation algorithm and evaluate its effectiveness and performance on four web service compositions.
  arXiv  Detail & Related papers  (2024-03-06T13:31:58Z)
• GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection? [50.53312866647302]
  HateCheck is a suite for testing fine-grained model functionalities on synthesized data. We propose GPT-HateCheck, a framework to generate more diverse and realistic functional tests from scratch. Crowd-sourced annotation demonstrates that the generated test cases are of high quality.
  arXiv  Detail & Related papers  (2024-02-23T10:02:01Z)
• Observation-based unit test generation at Meta [52.4716552057909]
  TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests.
  arXiv  Detail & Related papers  (2024-02-09T00:34:39Z)
• Metamorphic Testing of Image Captioning Systems via Image-Level Reduction [1.3225694028747141]
  In this paper, we propose REIC to perform metamorphic testing for IC systems with some image-level reduction transformations. With the image-level reduction transformations, REIC does not artificially manipulate any objects and hence can avoid generating unreal follow-up images.
  arXiv  Detail & Related papers  (2023-11-20T14:17:52Z)
• Deep anytime-valid hypothesis testing [29.273915933729057]
  We propose a general framework for constructing powerful, sequential hypothesis tests for nonparametric testing problems. We develop a principled approach of leveraging the representation capability of machine learning models within the testing-by-betting framework. Empirical results on synthetic and real-world datasets demonstrate that tests instantiated using our general framework are competitive against specialized baselines.
  arXiv  Detail & Related papers  (2023-10-30T09:46:19Z)
• Towards Automatic Generation of Amplified Regression Test Oracles [44.45138073080198]
  We propose a test oracle derivation approach to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour.
  arXiv  Detail & Related papers  (2023-07-28T12:38:44Z)
• Zero-shot Model Diagnosis [80.36063332820568]
  A common approach to evaluate deep learning models is to build a labeled test set with attributes of interest and assess how well it performs. This paper argues the case that Zero-shot Model Diagnosis (ZOOM) is possible without the need for a test set nor labeling.
  arXiv  Detail & Related papers  (2023-03-27T17:59:33Z)
• Using Metamorphic Relations to Verify and Enhance Artcode Classification [39.36253474867746]
  An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a set of predefined classes. An approach to software testing that alleviates the oracle problem is metamorphic testing (MT) This paper examines the problem of classifying images containing visually hidden markers called Artcodes, and applies MT to verify and enhance the trained classifiers.
  arXiv  Detail & Related papers  (2021-08-05T15:54:56Z)
• Automated identification of metamorphic test scenarios for an ocean-modeling application [0.802904964931021]
  We present work in progress for automated generation of metamorphic test scenarios using machine learning. Our application domain is ocean modeling, where test oracles often do not exist, but where symmetries of the simulated physical systems are known.
  arXiv  Detail & Related papers  (2020-09-03T10:03:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.