Single-step Adversarial training with Dropout Scheduling

• URL: http://arxiv.org/abs/2004.08628v1
• Date: Sat, 18 Apr 2020 14:14:00 GMT
• Title: Single-step Adversarial training with Dropout Scheduling
• Authors: Vivek B.S. and R. Venkatesh Babu
• Abstract summary: We show that models trained using single-step adversarial training method learn to prevent the generation of single-step adversaries. Models trained using proposed single-step adversarial training method are robust against both single-step and multi-step adversarial attacks.
• Score: 59.50324605982158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning models have shown impressive performance across a spectrum of computer vision applications including medical diagnosis and autonomous driving. One of the major concerns that these models face is their susceptibility to adversarial attacks. Realizing the importance of this issue, more researchers are working towards developing robust models that are less affected by adversarial attacks. Adversarial training method shows promising results in this direction. In adversarial training regime, models are trained with mini-batches augmented with adversarial samples. Fast and simple methods (e.g., single-step gradient ascent) are used for generating adversarial samples, in order to reduce computational complexity. It is shown that models trained using single-step adversarial training method (adversarial samples are generated using non-iterative method) are pseudo robust. Further, this pseudo robustness of models is attributed to the gradient masking effect. However, existing works fail to explain when and why gradient masking effect occurs during single-step adversarial training. In this work, (i) we show that models trained using single-step adversarial training method learn to prevent the generation of single-step adversaries, and this is due to over-fitting of the model during the initial stages of training, and (ii) to mitigate this effect, we propose a single-step adversarial training method with dropout scheduling. Unlike models trained using existing single-step adversarial training methods, models trained using the proposed single-step adversarial training method are robust against both single-step and multi-step adversarial attacks, and the performance is on par with models trained using computationally expensive multi-step adversarial training methods, in white-box and black-box settings.

Related papers

• Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks [69.54774045493227]
  A drawback of adversarial training is the computational overhead introduced by the generation of adversarial examples. We propose to exploit the interior building blocks of the model to improve efficiency. Compared with previous methods, our method not only reduces the training cost but also achieves better model robustness.
  arXiv  Detail & Related papers  (2023-10-24T01:36:20Z)
• CAT:Collaborative Adversarial Training [80.55910008355505]
  We propose a collaborative adversarial training framework to improve the robustness of neural networks. Specifically, we use different adversarial training methods to train robust models and let models interact with their knowledge during the training process. Cat achieves state-of-the-art adversarial robustness without using any additional data on CIFAR-10 under the Auto-Attack benchmark.
  arXiv  Detail & Related papers  (2023-03-27T05:37:43Z)
• Adversarial Fine-tune with Dynamically Regulated Adversary [27.034257769448914]
  In many real-world applications such as health diagnosis and autonomous surgical robotics, the standard performance is more valued over model robustness against such extremely malicious attacks. This work proposes a simple yet effective transfer learning-based adversarial training strategy that disentangles the negative effects of adversarial samples on model's standard performance. In addition, we introduce a training-friendly adversarial attack algorithm, which facilitates the boost of adversarial robustness without introducing significant training complexity.
  arXiv  Detail & Related papers  (2022-04-28T00:07:15Z)
• On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training [72.95029777394186]
  Adversarial training is a popular method to robustify models against adversarial attacks. We investigate this phenomenon from the perspective of training instances. We show that the decay in generalization performance of adversarial training is a result of the model's attempt to fit hard adversarial instances.
  arXiv  Detail & Related papers  (2021-12-14T12:19:24Z)
• Adaptive perturbation adversarial training: based on reinforcement learning [9.563820241076103]
  One of the shortcomings of adversarial training is that it will reduce the recognition accuracy of normal samples. Adaptive adversarial training is proposed to alleviate this problem. It uses marginal adversarial samples that are close to the decision boundary but does not cross the decision boundary for adversarial training.
  arXiv  Detail & Related papers  (2021-08-30T13:49:55Z)
• Multi-stage Optimization based Adversarial Training [16.295921205749934]
  We propose a Multi-stage Optimization based Adversarial Training (MOAT) method that periodically trains the model on mixed benign examples. Under similar amount of training overhead, the proposed MOAT exhibits better robustness than either single-step or multi-step adversarial training methods.
  arXiv  Detail & Related papers  (2021-06-26T07:59:52Z)
• Self-Progressing Robust Training [146.8337017922058]
  Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples. We propose a new framework called SPROUT, self-progressing robust training. Our results shed new light on scalable, effective and attack-independent robust training methods.
  arXiv  Detail & Related papers  (2020-12-22T00:45:24Z)
• Regularizers for Single-step Adversarial Training [49.65499307547198]
  We propose three types of regularizers that help to learn robust models using single-step adversarial training methods. Regularizers mitigate the effect of gradient masking by harnessing on properties that differentiate a robust model from that of a pseudo robust model.
  arXiv  Detail & Related papers  (2020-02-03T09:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.