Robust Single-step Adversarial Training with Regularizer
- URL: http://arxiv.org/abs/2102.03381v1
- Date: Fri, 5 Feb 2021 19:07:10 GMT
- Title: Robust Single-step Adversarial Training with Regularizer
- Authors: Lehui Xie, Yaopeng Wang, Jia-Li Yin, and Ximeng Liu
- Abstract summary: We propose a novel Fast Gradient Sign Method with PGD Regularization (FGSMPR) to boost the efficiency of adversarial training without catastrophic overfitting.
Experiments demonstrate that our proposed method can train a robust deep network for L$_infty$-perturbations with FGSM adversarial training.
- Score: 11.35007968593652
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: High cost of training time caused by multi-step adversarial example
generation is a major challenge in adversarial training. Previous methods try
to reduce the computational burden of adversarial training using single-step
adversarial example generation schemes, which can effectively improve the
efficiency but also introduce the problem of catastrophic overfitting, where
the robust accuracy against Fast Gradient Sign Method (FGSM) can achieve nearby
100\% whereas the robust accuracy against Projected Gradient Descent (PGD)
suddenly drops to 0\% over a single epoch. To address this problem, we propose
a novel Fast Gradient Sign Method with PGD Regularization (FGSMPR) to boost the
efficiency of adversarial training without catastrophic overfitting. Our core
idea is that single-step adversarial training can not learn robust internal
representations of FGSM and PGD adversarial examples. Therefore, we design a
PGD regularization term to encourage similar embeddings of FGSM and PGD
adversarial examples. The experiments demonstrate that our proposed method can
train a robust deep network for L$_\infty$-perturbations with FGSM adversarial
training and reduce the gap to multi-step adversarial training.
Related papers
- Reducing Adversarial Training Cost with Gradient Approximation [0.3916094706589679]
We propose a new and efficient adversarial training method, adversarial training with gradient approximation (GAAT) to reduce the cost of building up robust models.
Our proposed method saves up to 60% of the training time with comparable model test accuracy on datasets.
arXiv Detail & Related papers (2023-09-18T03:55:41Z) - Improving Fast Adversarial Training with Prior-Guided Knowledge [80.52575209189365]
We investigate the relationship between adversarial example quality and catastrophic overfitting by comparing the training processes of standard adversarial training and Fast adversarial training.
We find that catastrophic overfitting occurs when the attack success rate of adversarial examples becomes worse.
arXiv Detail & Related papers (2023-04-01T02:18:12Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Fast Adversarial Training with Adaptive Step Size [62.37203478589929]
We study the phenomenon from the perspective of training instances.
We propose a simple but effective method, Adversarial Training with Adaptive Step size (ATAS)
ATAS learns an instancewise adaptive step size that is inversely proportional to its gradient norm.
arXiv Detail & Related papers (2022-06-06T08:20:07Z) - Multi-stage Optimization based Adversarial Training [16.295921205749934]
We propose a Multi-stage Optimization based Adversarial Training (MOAT) method that periodically trains the model on mixed benign examples.
Under similar amount of training overhead, the proposed MOAT exhibits better robustness than either single-step or multi-step adversarial training methods.
arXiv Detail & Related papers (2021-06-26T07:59:52Z) - Self-Progressing Robust Training [146.8337017922058]
Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples.
We propose a new framework called SPROUT, self-progressing robust training.
Our results shed new light on scalable, effective and attack-independent robust training methods.
arXiv Detail & Related papers (2020-12-22T00:45:24Z) - Towards Rapid and Robust Adversarial Training with One-Step Attacks [0.0]
Adversarial training is the most successful method for increasing the robustness of neural networks against adversarial attacks.
We present two ideas that enable adversarial training with the computationally less expensive Fast Gradient Sign Method.
We show that noise injection in conjunction with FGSM-based adversarial training achieves comparable results to adversarial training with PGD while being considerably faster.
arXiv Detail & Related papers (2020-02-24T07:28:43Z) - Improving the affordability of robustness training for DNNs [11.971637253035107]
We show that the initial phase of adversarial training is redundant and can be replaced with natural training which significantly improves the computational efficiency.
We show that our proposed method can reduce the training time by a factor of up to 2.5 with comparable or better model test accuracy and generalization on various strengths of adversarial attacks.
arXiv Detail & Related papers (2020-02-11T07:29:45Z) - Fast is better than free: Revisiting adversarial training [86.11788847990783]
We show that it is possible to train empirically robust models using a much weaker and cheaper adversary.
We identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail.
arXiv Detail & Related papers (2020-01-12T20:30:22Z) - Efficient Adversarial Training with Transferable Adversarial Examples [58.62766224452761]
We show that there is high transferability between models from neighboring epochs in the same training process.
We propose a novel method, Adversarial Training with Transferable Adversarial Examples (ATTA) that can enhance the robustness of trained models.
arXiv Detail & Related papers (2019-12-27T03:05:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.