GIST: Generated Inputs Sets Transferability in Deep Learning

• URL: http://arxiv.org/abs/2311.00801v3
• Date: Mon, 20 May 2024 13:33:08 GMT
• Title: GIST: Generated Inputs Sets Transferability in Deep Learning
• Authors: Florian Tambon, Foutse Khomh, Giuliano Antoniol,
• Abstract summary: GIST (Generated Inputs Sets Transferability) is a novel approach for the efficient transfer of test sets. This paper introduces GIST, a novel approach for the efficient transfer of test sets.
• Score: 12.147546375400749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To foster the verifiability and testability of Deep Neural Networks (DNN), an increasing number of methods for test case generation techniques are being developed. When confronted with testing DNN models, the user can apply any existing test generation technique. However, it needs to do so for each technique and each DNN model under test, which can be expensive. Therefore, a paradigm shift could benefit this testing process: rather than regenerating the test set independently for each DNN model under test, we could transfer from existing DNN models. This paper introduces GIST (Generated Inputs Sets Transferability), a novel approach for the efficient transfer of test sets. Given a property selected by a user (e.g., neurons covered, faults), GIST enables the selection of good test sets from the point of view of this property among available test sets. This allows the user to recover similar properties on the transferred test sets as he would have obtained by generating the test set from scratch with a test cases generation technique. Experimental results show that GIST can select effective test sets for the given property to transfer. Moreover, GIST scales better than reapplying test case generation techniques from scratch on DNN models under test.

Related papers

• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [52.98590762456236]
  Training-free test-time dynamic adapter (TDA) is a promising approach to address this issue. We propose a simple yet effective method for DistributiOnal Test-time Adaptation (Dota) Dota continually estimates the distributions of test samples, allowing the model to continually adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Robust Black-box Testing of Deep Neural Networks using Co-Domain Coverage [18.355332126489756]
  Rigorous testing of machine learning models is necessary for trustworthy deployments. We present a novel black-box approach for generating test-suites for robust testing of deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2024-08-13T09:42:57Z)
• Novel Deep Neural Network Classifier Characterization Metrics with Applications to Dataless Evaluation [1.6574413179773757]
  In this work, we evaluate a Deep Neural Network (DNN) classifier's training quality without any example dataset. Our empirical study of the proposed method for ResNet18, trained with CAFIR10 and CAFIR100 datasets, confirms that data-less evaluation of DNN classifiers is indeed possible.
  arXiv  Detail & Related papers  (2024-07-17T20:40:46Z)
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [143.14128737978342]
  Test-time adaptation, an emerging paradigm, has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions. Recent progress in this paradigm highlights the significant benefits of utilizing unlabeled data for training self-adapted models prior to inference.
  arXiv  Detail & Related papers  (2023-03-27T16:32:21Z)
• TeST: Test-time Self-Training under Distribution Shift [99.68465267994783]
  Test-Time Self-Training (TeST) is a technique that takes as input a model trained on some source data and a novel data distribution at test time. We find that models adapted using TeST significantly improve over baseline test-time adaptation algorithms.
  arXiv  Detail & Related papers  (2022-09-23T07:47:33Z)
• TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
  We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples. We show the success of our method on the common benchmark dataset CIFAR10-C.
  arXiv  Detail & Related papers  (2022-05-18T05:43:06Z)
• Efficient Test-Time Model Adaptation without Forgetting [60.36499845014649]
  Test-time adaptation seeks to tackle potential distribution shifts between training and testing data. We propose an active sample selection criterion to identify reliable and non-redundant samples. We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
  arXiv  Detail & Related papers  (2022-04-06T06:39:40Z)
• Machine Learning Testing in an ADAS Case Study Using Simulation-Integrated Bio-Inspired Search-Based Testing [7.5828169434922]
  Deeper generates failure-revealing test scenarios for testing a deep neural network-based lane-keeping system. In the newly proposed version, we utilize a new set of bio-inspired search algorithms, genetic algorithm (GA), $(mu+lambda)$ and $(mu,lambda)$ evolution strategies (ES), and particle swarm optimization (PSO) Our evaluation shows the newly proposed test generators in Deeper represent a considerable improvement on the previous version.
  arXiv  Detail & Related papers  (2022-03-22T20:27:40Z)
• Distribution-Aware Testing of Neural Networks Using Generative Models [5.618419134365903]
  The reliability of software that has a Deep Neural Network (DNN) as a component is urgently important. We show that three recent testing techniques generate significant number of invalid test inputs. We propose a technique to incorporate the valid input space of the DNN model under test in the test generation process.
  arXiv  Detail & Related papers  (2021-02-26T17:18:21Z)
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design [63.48989885374238]
  When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testing algorithms that can operate in a noisy setting.
  arXiv  Detail & Related papers  (2020-04-26T23:41:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.