Efficient Robust Training via Backward Smoothing
- URL: http://arxiv.org/abs/2010.01278v2
- Date: Thu, 30 Dec 2021 07:50:54 GMT
- Title: Efficient Robust Training via Backward Smoothing
- Authors: Jinghui Chen and Yu Cheng and Zhe Gan and Quanquan Gu and Jingjing Liu
- Abstract summary: Adversarial training is the most effective strategy in defending against adversarial examples.
It suffers from high computational costs due to the iterative adversarial attacks in each training step.
Recent studies show that it is possible to achieve fast Adversarial Training by performing a single-step attack.
- Score: 125.91185167854262
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Adversarial training is so far the most effective strategy in defending
against adversarial examples. However, it suffers from high computational costs
due to the iterative adversarial attacks in each training step. Recent studies
show that it is possible to achieve fast Adversarial Training by performing a
single-step attack with random initialization. However, such an approach still
lags behind state-of-the-art adversarial training algorithms on both stability
and model robustness. In this work, we develop a new understanding towards Fast
Adversarial Training, by viewing random initialization as performing randomized
smoothing for better optimization of the inner maximization problem. Following
this new perspective, we also propose a new initialization strategy, backward
smoothing, to further improve the stability and model robustness over
single-step robust training methods. Experiments on multiple benchmarks
demonstrate that our method achieves similar model robustness as the original
TRADES method while using much less training time ($\sim$3x improvement with
the same training schedule).
Related papers
- Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks [11.389689242531327]
Adversarial training is one of the most effective methods for enhancing model robustness.
Previous approaches primarily use static ground truth for adversarial training, but this often causes robust overfitting.
We propose a dynamic label adversarial training (DYNAT) algorithm that enables the target model to gain robustness from the guide model's decisions.
arXiv Detail & Related papers (2024-08-23T14:25:12Z) - Fast Propagation is Better: Accelerating Single-Step Adversarial
Training via Sampling Subnetworks [69.54774045493227]
A drawback of adversarial training is the computational overhead introduced by the generation of adversarial examples.
We propose to exploit the interior building blocks of the model to improve efficiency.
Compared with previous methods, our method not only reduces the training cost but also achieves better model robustness.
arXiv Detail & Related papers (2023-10-24T01:36:20Z) - Adversarial Coreset Selection for Efficient Robust Training [11.510009152620666]
We show how selecting a small subset of training data provides a principled approach to reducing the time complexity of robust training.
We conduct extensive experiments to demonstrate that our approach speeds up adversarial training by 2-3 times.
arXiv Detail & Related papers (2022-09-13T07:37:53Z) - Boosting Fast Adversarial Training with Learnable Adversarial
Initialization [79.90495058040537]
Adrial training (AT) has been demonstrated to be effective in improving model robustness by leveraging adversarial examples for training.
To boost training efficiency, fast gradient sign method (FGSM) is adopted in fast AT methods by calculating gradient only once.
arXiv Detail & Related papers (2021-10-11T05:37:00Z) - Self-Progressing Robust Training [146.8337017922058]
Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples.
We propose a new framework called SPROUT, self-progressing robust training.
Our results shed new light on scalable, effective and attack-independent robust training methods.
arXiv Detail & Related papers (2020-12-22T00:45:24Z) - Improved Adversarial Training via Learned Optimizer [101.38877975769198]
We propose a framework to improve the robustness of adversarial training models.
By co-training's parameters model's weights, the proposed framework consistently improves robustness and steps adaptively for update directions.
arXiv Detail & Related papers (2020-04-25T20:15:53Z) - Single-step Adversarial training with Dropout Scheduling [59.50324605982158]
We show that models trained using single-step adversarial training method learn to prevent the generation of single-step adversaries.
Models trained using proposed single-step adversarial training method are robust against both single-step and multi-step adversarial attacks.
arXiv Detail & Related papers (2020-04-18T14:14:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.