SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation

• URL: http://arxiv.org/abs/2005.09296v1
• Date: Tue, 19 May 2020 09:06:21 GMT
• Title: SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation
• Authors: Sungmin Kang (1), Robert Feldt (2), Shin Yoo (1) ((1) School of Computing KAIST, (2) Chalmers University)
• Abstract summary: Testing of Deep Neural Networks (DNNs) has become increasingly important as DNNs are widely adopted by safety critical systems. Current testing techniques for DNNs depend on small local perturbations to existing inputs. We propose new ways to search not over the entire image space, but rather over a plausible input space that resembles the true training distribution.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The testing of Deep Neural Networks (DNNs) has become increasingly important as DNNs are widely adopted by safety critical systems. While many test adequacy criteria have been suggested, automated test input generation for many types of DNNs remains a challenge because the raw input space is too large to randomly sample or to navigate and search for plausible inputs. Consequently, current testing techniques for DNNs depend on small local perturbations to existing inputs, based on the metamorphic testing principle. We propose new ways to search not over the entire image space, but rather over a plausible input space that resembles the true training distribution. This space is constructed using Variational Autoencoders (VAEs), and navigated through their latent vector space. We show that this space helps efficiently produce test inputs that can reveal information about the robustness of DNNs when dealing with realistic tests, opening the field to meaningful exploration through the space of highly structured images.

Related papers

• NAS-BNN: Neural Architecture Search for Binary Neural Networks [55.058512316210056]
  We propose a novel neural architecture search scheme for binary neural networks, named NAS-BNN. Our discovered binary model family outperforms previous BNNs for a wide range of operations (OPs) from 20M to 200M. In addition, we validate the transferability of these searched BNNs on the object detection task, and our binary detectors with the searched BNNs achieve a novel state-of-the-art result, e.g., 31.6% mAP with 370M OPs, on MS dataset.
  arXiv  Detail & Related papers  (2024-08-28T02:17:58Z)
• Search-based DNN Testing and Retraining with GAN-enhanced Simulations [2.362412515574206]
  In safety-critical systems, Deep Neural Networks (DNNs) are becoming a key component for computer vision tasks. We propose to combine meta-heuristic search, used to explore the input space using simulators, with Generative Adversarial Networks (GANs) to transform the data generated by simulators into realistic input images.
  arXiv  Detail & Related papers  (2024-06-19T09:05:16Z)
• DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks [0.6249768559720121]
  DeepGD is a black-box multi-objective test selection approach for Deep neural networks (DNNs) It reduces the cost of labeling by prioritizing the selection of test inputs with high fault revealing power from large unlabeled datasets.
  arXiv  Detail & Related papers  (2023-03-08T20:33:09Z)
• The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networks [94.63547069706459]
  #DNN-Verification problem involves counting the number of input configurations of a DNN that result in a violation of a safety property. We propose a novel approach that returns the exact count of violations. We present experimental results on a set of safety-critical benchmarks.
  arXiv  Detail & Related papers  (2023-01-17T18:32:01Z)
• DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessment [10.18462284491991]
  Quantization is one of the most applied Deep Neural Network (DNN) compression strategies. We present DiverGet, a search-based testing framework for quantization assessment. We evaluate the performance of DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing images.
  arXiv  Detail & Related papers  (2022-07-13T15:27:51Z)
• Fully Spiking Variational Autoencoder [66.58310094608002]
  Spiking neural networks (SNNs) can be run on neuromorphic devices with ultra-high speed and ultra-low energy consumption. In this study, we build a variational autoencoder (VAE) with SNN to enable image generation.
  arXiv  Detail & Related papers  (2021-09-26T06:10:14Z)
• Distribution-Aware Testing of Neural Networks Using Generative Models [5.618419134365903]
  The reliability of software that has a Deep Neural Network (DNN) as a component is urgently important. We show that three recent testing techniques generate significant number of invalid test inputs. We propose a technique to incorporate the valid input space of the DNN model under test in the test generation process.
  arXiv  Detail & Related papers  (2021-02-26T17:18:21Z)
• A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art performance on numerous applications. It is difficult to tell beforehand if a DNN receiving an input will deliver the correct output since their decision criteria are usually nontransparent. This survey connects the three fields within the larger framework of investigating the generalization performance of machine learning methods and in particular DNNs.
  arXiv  Detail & Related papers  (2020-08-21T09:12:52Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• GraN: An Efficient Gradient-Norm Based Detector for Adversarial and Misclassified Examples [77.99182201815763]
  Deep neural networks (DNNs) are vulnerable to adversarial examples and other data perturbations. GraN is a time- and parameter-efficient method that is easily adaptable to any DNN. GraN achieves state-of-the-art performance on numerous problem set-ups.
  arXiv  Detail & Related papers  (2020-04-20T10:09:27Z)
• Broad Area Search and Detection of Surface-to-Air Missile Sites Using Spatial Fusion of Component Object Detections from Deep Neural Networks [7.24548168665473]
  Deep Neural Network (DNN) detections of multiple or component objects can be spatially fused to improve the search, detection, and retrieval (ranking) of a larger complex feature. We demonstrate the utility of this approach for broad area search and detection of Surface-to-Air Missile (SAM) sites over a 90,000 km2 study area in SE China.
  arXiv  Detail & Related papers  (2020-03-23T22:10:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.