HOMRS: High Order Metamorphic Relations Selector for Deep Neural
Networks
- URL: http://arxiv.org/abs/2107.04863v1
- Date: Sat, 10 Jul 2021 15:40:12 GMT
- Title: HOMRS: High Order Metamorphic Relations Selector for Deep Neural
Networks
- Authors: Florian Tambon, Giulio Antoniol and Foutse Khomh
- Abstract summary: We present HOMRS, an approach to boost metamorphic testing by automatically building a small optimized set of high order metamorphic relations.
Five raters manually labeled a pool of images before and after high order transformation; Fleiss' Kappa and statistical tests confirmed that they are metamorphic properties.
- Score: 7.369475193451258
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Neural Networks (DNN) applications are increasingly becoming a part of
our everyday life, from medical applications to autonomous cars. Traditional
validation of DNN relies on accuracy measures, however, the existence of
adversarial examples has highlighted the limitations of these accuracy
measures, raising concerns especially when DNN are integrated into
safety-critical systems. In this paper, we present HOMRS, an approach to boost
metamorphic testing by automatically building a small optimized set of high
order metamorphic relations from an initial set of elementary metamorphic
relations. HOMRS' backbone is a multi-objective search; it exploits ideas drawn
from traditional systems testing such as code coverage, test case, and path
diversity. We applied HOMRS to LeNet5 DNN with MNIST dataset and we report
evidence that it builds a small but effective set of high order transformations
achieving a 95% kill ratio. Five raters manually labeled a pool of images
before and after high order transformation; Fleiss' Kappa and statistical tests
confirmed that they are metamorphic properties. HOMRS built-in relations are
also effective to confront adversarial or out-of-distribution examples; HOMRS
detected 92% of randomly sampled out-of-distribution images. HOMRS
transformations are also suitable for online real-time use.
Related papers
- A lightweight residual network for unsupervised deformable image registration [2.7309692684728617]
We propose a residual U-Net with embedded parallel dilated-convolutional blocks to enhance the receptive field.
The proposed method is evaluated on inter-patient and atlas-based datasets.
arXiv Detail & Related papers (2024-06-14T07:20:49Z) - Improving the Robustness of Quantized Deep Neural Networks to White-Box
Attacks using Stochastic Quantization and Information-Theoretic Ensemble
Training [1.6098666134798774]
Most real-world applications that employ deep neural networks (DNNs) quantize them to low precision to reduce the compute needs.
We present a method to improve the robustness of quantized DNNs to white-box adversarial attacks.
arXiv Detail & Related papers (2023-11-30T17:15:58Z) - Heterogenous Memory Augmented Neural Networks [84.29338268789684]
We introduce a novel heterogeneous memory augmentation approach for neural networks.
By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead.
We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
arXiv Detail & Related papers (2023-10-17T01:05:28Z) - A Geometrical Approach to Evaluate the Adversarial Robustness of Deep
Neural Networks [52.09243852066406]
Adversarial Converging Time Score (ACTS) measures the converging time as an adversarial robustness metric.
We validate the effectiveness and generalization of the proposed ACTS metric against different adversarial attacks on the large-scale ImageNet dataset.
arXiv Detail & Related papers (2023-10-10T09:39:38Z) - Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement
Learning [53.00683059396803]
Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images.
We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy.
Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
arXiv Detail & Related papers (2023-10-06T10:40:46Z) - Deep Multi-Threshold Spiking-UNet for Image Processing [51.88730892920031]
This paper introduces the novel concept of Spiking-UNet for image processing, which combines the power of Spiking Neural Networks (SNNs) with the U-Net architecture.
To achieve an efficient Spiking-UNet, we face two primary challenges: ensuring high-fidelity information propagation through the network via spikes and formulating an effective training strategy.
Experimental results show that, on image segmentation and denoising, our Spiking-UNet achieves comparable performance to its non-spiking counterpart.
arXiv Detail & Related papers (2023-07-20T16:00:19Z) - Unfolding Local Growth Rate Estimates for (Almost) Perfect Adversarial
Detection [22.99930028876662]
Convolutional neural networks (CNN) define the state-of-the-art solution on many perceptual tasks.
Current CNN approaches largely remain vulnerable against adversarial perturbations of the input that have been crafted specifically to fool the system.
We propose a simple and light-weight detector, which leverages recent findings on the relation between networks' local intrinsic dimensionality (LID) and adversarial attacks.
arXiv Detail & Related papers (2022-12-13T17:51:32Z) - DiverGet: A Search-Based Software Testing Approach for Deep Neural
Network Quantization Assessment [10.18462284491991]
Quantization is one of the most applied Deep Neural Network (DNN) compression strategies.
We present DiverGet, a search-based testing framework for quantization assessment.
We evaluate the performance of DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing images.
arXiv Detail & Related papers (2022-07-13T15:27:51Z) - Comparative Analysis of Interval Reachability for Robust Implicit and
Feedforward Neural Networks [64.23331120621118]
We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs)
INNs are a class of implicit learning models that use implicit equations as layers.
We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
arXiv Detail & Related papers (2022-04-01T03:31:27Z) - An Efficient Confidence Measure-Based Evaluation Metric for Breast
Cancer Screening Using Bayesian Neural Networks [3.834509400202395]
We propose a confidence measure-based evaluation metric for breast cancer screening.
We show that our confidence tuning results in increased accuracy with a reduced set of images with high confidence when compared to the baseline transfer learning.
arXiv Detail & Related papers (2020-08-12T20:34:14Z) - GraN: An Efficient Gradient-Norm Based Detector for Adversarial and
Misclassified Examples [77.99182201815763]
Deep neural networks (DNNs) are vulnerable to adversarial examples and other data perturbations.
GraN is a time- and parameter-efficient method that is easily adaptable to any DNN.
GraN achieves state-of-the-art performance on numerous problem set-ups.
arXiv Detail & Related papers (2020-04-20T10:09:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.