Understanding Local Robustness of Deep Neural Networks under Natural Variations

• URL: http://arxiv.org/abs/2010.04821v2
• Date: Sat, 23 Jan 2021 02:46:18 GMT
• Title: Understanding Local Robustness of Deep Neural Networks under Natural Variations
• Authors: Ziyuan Zhong, Yuchi Tian, Baishakhi Ray
• Abstract summary: Deep Neural Networks (DNNs) are being deployed in a wide range of settings today. Recent research has shown that DNNs can be brittle to even slight variations of the input data.
• Score: 18.638234554232994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Neural Networks (DNNs) are being deployed in a wide range of settings today, from safety-critical applications like autonomous driving to commercial applications involving image classifications. However, recent research has shown that DNNs can be brittle to even slight variations of the input data. Therefore, rigorous testing of DNNs has gained widespread attention. While DNN robustness under norm-bound perturbation got significant attention over the past few years, our knowledge is still limited when natural variants of the input images come. These natural variants, e.g. a rotated or a rainy version of the original input, are especially concerning as they can occur naturally in the field without any active adversary and may lead to undesirable consequences. Thus, it is important to identify the inputs whose small variations may lead to erroneous DNN behaviors. The very few studies that looked at DNN's robustness under natural variants, however, focus on estimating the overall robustness of DNNs across all the test data rather than localizing such error-producing points. This work aims to bridge this gap. To this end, we study the local per-input robustness properties of the DNNs and leverage those properties to build a white-box (DeepRobust-W) and a black-box (DeepRobust-B) tool to automatically identify the non-robust points. Our evaluation of these methods on three DNN models spanning three widely used image classification datasets shows that they are effective in flagging points of poor robustness. In particular, DeepRobust-W and DeepRobust-B are able to achieve an F1 score of up to 91.4% and 99.1%, respectively. We further show that DeepRobust-W can be applied to a regression problem in another domain. Our evaluation on three self-driving car models demonstrates that DeepRobust-W is effective in identifying points of poor robustness with F1 score up to 78.9%.

Related papers

• Harnessing Neuron Stability to Improve DNN Verification [42.65507402735545]
  We present VeriStable, a novel extension of recently proposed DPLL-based constraint DNN verification approach. We evaluate the effectiveness of VeriStable across a range of challenging benchmarks including fully-connected feed networks (FNNs), convolutional neural networks (CNNs) and residual networks (ResNets) Preliminary results show that VeriStable is competitive and outperforms state-of-the-art verification tools, including $alpha$-$beta$-CROWN and MN-BaB, the first and second performers of the VNN-COMP, respectively.
  arXiv  Detail & Related papers  (2024-01-19T23:48:04Z)
• SAfER: Layer-Level Sensitivity Assessment for Efficient and Robust Neural Network Inference [20.564198591600647]
  Deep neural networks (DNNs) demonstrate outstanding performance across most computer vision tasks. Some critical applications, such as autonomous driving or medical imaging, also require investigation into their behavior. DNN attribution consists in studying the relationship between the predictions of a DNN and its inputs.
  arXiv  Detail & Related papers  (2023-08-09T07:45:51Z)
• gRoMA: a Tool for Measuring the Global Robustness of Deep Neural Networks [3.2228025627337864]
  Deep neural networks (DNNs) are at the forefront of cutting-edge technology, and have been achieving remarkable performance in a variety of complex tasks. Their integration into safety-critical systems, such as in the aerospace or automotive domains, poses a significant challenge due to the threat of adversarial inputs. Here, we present gRoMA, an innovative and scalable tool that implements a probabilistic approach to measure the global categorial robustness of a DNN.
  arXiv  Detail & Related papers  (2023-01-05T20:45:23Z)
• Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead [67.87678914831477]
  Deep Neural Networks (DNNs) enable a wide series of technological advancements. Recent findings indicate that transient hardware faults may corrupt the models prediction dramatically. In this work, we propose to tackle the reliability issue both at training and model design time.
  arXiv  Detail & Related papers  (2022-05-28T13:09:30Z)
• Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering [0.9590956574213348]
  We propose SAFE, a black-box approach to automatically characterize the root causes of DNN errors. It relies on a transfer learning model pre-trained on ImageNet to extract the features from error-inducing images. It then applies a density-based clustering algorithm to detect arbitrary shaped clusters of images modeling plausible causes of error.
  arXiv  Detail & Related papers  (2022-01-13T17:02:57Z)
• Robustness of Bayesian Neural Networks to White-Box Adversarial Attacks [55.531896312724555]
  Bayesian Networks (BNNs) are robust and adept at handling adversarial attacks by incorporating randomness. We create our BNN model, called BNN-DenseNet, by fusing Bayesian inference (i.e., variational Bayes) to the DenseNet architecture. An adversarially-trained BNN outperforms its non-Bayesian, adversarially-trained counterpart in most experiments.
  arXiv  Detail & Related papers  (2021-11-16T16:14:44Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning [61.93730166203915]
  We argue that traditional notions of fairness are not sufficient when the model is vulnerable to adversarial attacks. We show that measuring robustness bias is a challenging task for DNNs and propose two methods to measure this form of bias.
  arXiv  Detail & Related papers  (2020-06-17T22:22:24Z)
• GraN: An Efficient Gradient-Norm Based Detector for Adversarial and Misclassified Examples [77.99182201815763]
  Deep neural networks (DNNs) are vulnerable to adversarial examples and other data perturbations. GraN is a time- and parameter-efficient method that is easily adaptable to any DNN. GraN achieves state-of-the-art performance on numerous problem set-ups.
  arXiv  Detail & Related papers  (2020-04-20T10:09:27Z)
• Supporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learning [1.6414392145248926]
  We propose HUDD, an approach that automatically supports the identification of root causes for DNN errors. HUDD identifies root causes by applying a clustering algorithm to heatmaps capturing the relevance of every DNN neuron on the outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters.
  arXiv  Detail & Related papers  (2020-02-03T16:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.