Fast Training of Deep Neural Networks Robust to Adversarial Perturbations

• URL: http://arxiv.org/abs/2007.03832v1
• Date: Wed, 8 Jul 2020 00:35:39 GMT
• Title: Fast Training of Deep Neural Networks Robust to Adversarial Perturbations
• Authors: Justin Goodwin, Olivia Brown, Victoria Helus
• Abstract summary: We show that a fast approximation to adversarial training shows promise for reducing training time and maintaining robustness. Fast adversarial training is a promising approach that will provide increased security and explainability in machine learning applications.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks are capable of training fast and generalizing well within many domains. Despite their promising performance, deep networks have shown sensitivities to perturbations of their inputs (e.g., adversarial examples) and their learned feature representations are often difficult to interpret, raising concerns about their true capability and trustworthiness. Recent work in adversarial training, a form of robust optimization in which the model is optimized against adversarial examples, demonstrates the ability to improve performance sensitivities to perturbations and yield feature representations that are more interpretable. Adversarial training, however, comes with an increased computational cost over that of standard (i.e., nonrobust) training, rendering it impractical for use in large-scale problems. Recent work suggests that a fast approximation to adversarial training shows promise for reducing training time and maintaining robustness in the presence of perturbations bounded by the infinity norm. In this work, we demonstrate that this approach extends to the Euclidean norm and preserves the human-aligned feature representations that are common for robust models. Additionally, we show that using a distributed training scheme can further reduce the time to train robust deep networks. Fast adversarial training is a promising approach that will provide increased security and explainability in machine learning applications for which robust optimization was previously thought to be impractical.

Related papers

• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data [38.44734564565478]
  We provide a theoretical understanding of adversarial examples and adversarial training algorithms from the perspective of feature learning theory. We show that the adversarial training method can provably strengthen the robust feature learning and suppress the non-robust feature learning.
  arXiv  Detail & Related papers  (2024-10-11T03:59:49Z)
• Accurate Neural Network Pruning Requires Rethinking Sparse Optimization [87.90654868505518]
  We show the impact of high sparsity on model training using the standard computer vision and natural language processing sparsity benchmarks. We provide new approaches for mitigating this issue for both sparse pre-training of vision models and sparse fine-tuning of language models.
  arXiv  Detail & Related papers  (2023-08-03T21:49:14Z)
• Improved Adversarial Training Through Adaptive Instance-wise Loss Smoothing [5.1024659285813785]
  Adversarial training has been the most successful defense against such adversarial attacks. We propose a new adversarial training method: Instance-adaptive Smoothness Enhanced Adversarial Training. Our method achieves state-of-the-art robustness against $ell_infty$-norm constrained attacks.
  arXiv  Detail & Related papers  (2023-03-24T15:41:40Z)
• Addressing Mistake Severity in Neural Networks with Semantic Knowledge [0.0]
  Most robust training techniques aim to improve model accuracy on perturbed inputs. As an alternate form of robustness, we aim to reduce the severity of mistakes made by neural networks in challenging conditions. We leverage current adversarial training methods to generate targeted adversarial attacks during the training process. Results demonstrate that our approach performs better with respect to mistake severity compared to standard and adversarially trained models.
  arXiv  Detail & Related papers  (2022-11-21T22:01:36Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Enhancing Adversarial Training with Feature Separability [52.39305978984573]
  We introduce a new concept of adversarial training graph (ATG) with which the proposed adversarial training with feature separability (ATFS) enables to boost the intra-class feature similarity and increase inter-class feature variance. Through comprehensive experiments, we demonstrate that the proposed ATFS framework significantly improves both clean and robust performance.
  arXiv  Detail & Related papers  (2022-05-02T04:04:23Z)
• Second Order Optimization for Adversarial Robustness and Interpretability [6.700873164609009]
  We propose a novel regularizer which incorporates first and second order information via a quadratic approximation to the adversarial loss. It is shown that using only a single iteration in our regularizer achieves stronger robustness than prior gradient and curvature regularization schemes. It retains the interesting facet of AT that networks learn features which are well-aligned with human perception.
  arXiv  Detail & Related papers  (2020-09-10T15:05:14Z)
• Optimizing Information Loss Towards Robust Neural Networks [0.0]
  Neural Networks (NNs) are vulnerable to adversarial examples. We present a new training approach we call textitentropic retraining. Based on an information-theoretic-inspired analysis, entropic retraining mimics the effects of adversarial training without the need of the laborious generation of adversarial examples.
  arXiv  Detail & Related papers  (2020-08-07T10:12:31Z)
• Towards Understanding Fast Adversarial Training [91.8060431517248]
  We conduct experiments to understand the behavior of fast adversarial training. We show the key to its success is the ability to recover from overfitting to weak attacks.
  arXiv  Detail & Related papers  (2020-06-04T18:19:43Z)
• Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training [56.82000424924979]
  We propose a conditional normalization module to adapt networks when conditioned on input samples. Our adaptive networks, once adversarially trained, can outperform their non-adaptive counterparts on both clean validation accuracy and robustness.
  arXiv  Detail & Related papers  (2020-05-30T23:23:56Z)
• Towards Achieving Adversarial Robustness by Enforcing Feature Consistency Across Bit Planes [51.31334977346847]
  We train networks to form coarse impressions based on the information in higher bit planes, and use the lower bit planes only to refine their prediction. We demonstrate that, by imposing consistency on the representations learned across differently quantized images, the adversarial robustness of networks improves significantly.
  arXiv  Detail & Related papers  (2020-04-01T09:31:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.