A Theoretical Perspective on Subnetwork Contributions to Adversarial Robustness

• URL: http://arxiv.org/abs/2307.03803v1
• Date: Fri, 7 Jul 2023 19:16:59 GMT
• Title: A Theoretical Perspective on Subnetwork Contributions to Adversarial Robustness
• Authors: Jovon Craig, Josh Andle, Theodore S. Nowak, Salimeh Yasaei Sekeh
• Abstract summary: This paper investigates how the adversarial robustness of a subnetwork contributes to the robustness of the entire network. Experiments show the ability of a robust subnetwork to promote full-network robustness, and investigate the layer-wise dependencies required for this full-network robustness to be achieved.
• Score: 2.064612766965483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The robustness of deep neural networks (DNNs) against adversarial attacks has been studied extensively in hopes of both better understanding how deep learning models converge and in order to ensure the security of these models in safety-critical applications. Adversarial training is one approach to strengthening DNNs against adversarial attacks, and has been shown to offer a means for doing so at the cost of applying computationally expensive training methods to the entire model. To better understand these attacks and facilitate more efficient adversarial training, in this paper we develop a novel theoretical framework that investigates how the adversarial robustness of a subnetwork contributes to the robustness of the entire network. To do so we first introduce the concept of semirobustness, which is a measure of the adversarial robustness of a subnetwork. Building on this concept, we then provide a theoretical analysis to show that if a subnetwork is semirobust and there is a sufficient dependency between it and each subsequent layer in the network, then the remaining layers are also guaranteed to be robust. We validate these findings empirically across multiple DNN architectures, datasets, and adversarial attacks. Experiments show the ability of a robust subnetwork to promote full-network robustness, and investigate the layer-wise dependencies required for this full-network robustness to be achieved.

Related papers

• Robust NAS under adversarial training: benchmark, theory, and beyond [55.51199265630444]
  We release a comprehensive data set that encompasses both clean accuracy and robust accuracy for a vast array of adversarially trained networks. We also establish a generalization theory for searching architecture in terms of clean accuracy and robust accuracy under multi-objective adversarial training.
  arXiv  Detail & Related papers  (2024-03-19T20:10:23Z)
• Understanding Robust Learning through the Lens of Representation Similarities [37.66877172364004]
  robustness to adversarial examples has emerged as a desirable property for deep neural networks (DNNs) In this paper, we aim to understand how the properties of representations learned by robust training differ from those obtained from standard, non-robust training.
  arXiv  Detail & Related papers  (2022-06-20T16:06:20Z)
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks [98.21130211336964]
  Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. In this paper, we investigate the impact of network width and depth on the robustness of adversarially trained DNNs.
  arXiv  Detail & Related papers  (2021-10-07T23:13:33Z)
• SoK: Certified Robustness for Deep Neural Networks [13.10665264010575]
  Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial attacks. In this paper, we systematize certifiably robust approaches and related practical and theoretical implications. We also provide the first comprehensive benchmark on existing robustness verification and training approaches on different datasets.
  arXiv  Detail & Related papers  (2020-09-09T07:00:55Z)
• Improving Adversarial Robustness by Enforcing Local and Global Compactness [19.8818435601131]
  Adversary training is the most successful method that consistently resists a wide range of attacks. We propose the Adversary Divergence Reduction Network which enforces local/global compactness and the clustering assumption. The experimental results demonstrate that augmenting adversarial training with our proposed components can further improve the robustness of the network.
  arXiv  Detail & Related papers  (2020-07-10T00:43:06Z)
• A general framework for defining and optimizing robustness [74.67016173858497]
  We propose a rigorous and flexible framework for defining different types of robustness properties for classifiers. Our concept is based on postulates that robustness of a classifier should be considered as a property that is independent of accuracy. We develop a very general robustness framework that is applicable to any type of classification model.
  arXiv  Detail & Related papers  (2020-06-19T13:24:20Z)
• Rethinking Clustering for Robustness [56.14672993686335]
  ClusTR is a clustering-based and adversary-free training framework to learn robust models. textitClusTR outperforms adversarially-trained networks by up to $4%$ under strong PGD attacks.
  arXiv  Detail & Related papers  (2020-06-13T16:55:51Z)
• Towards Practical Lottery Ticket Hypothesis for Adversarial Training [78.30684998080346]
  We show there exists a subset of the aforementioned sub-networks that converge significantly faster during the training process. As a practical application of our findings, we demonstrate that such sub-networks can help in cutting down the total time of adversarial training.
  arXiv  Detail & Related papers  (2020-03-06T03:11:52Z)
• HYDRA: Pruning Adversarially Robust Neural Networks [58.061681100058316]
  Deep learning faces two key challenges: lack of robustness against adversarial attacks and large neural network size. We propose to make pruning techniques aware of the robust training objective and let the training objective guide the search for which connections to prune. We demonstrate that our approach, titled HYDRA, achieves compressed networks with state-of-the-art benign and robust accuracy, simultaneously.
  arXiv  Detail & Related papers  (2020-02-24T19:54:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.