Towards Speeding up Adversarial Training in Latent Spaces
- URL: http://arxiv.org/abs/2102.00662v1
- Date: Mon, 1 Feb 2021 06:30:32 GMT
- Title: Towards Speeding up Adversarial Training in Latent Spaces
- Authors: Yaguan Qian, Qiqi Shao, Tengteng Yao, Bin Wang, Shaoning Zeng,
Zhaoquan Gu and Wassim Swaileh
- Abstract summary: We propose a novel adversarial training method that does not need to generate real adversarial examples.
We gain a deep insight into the existence of Endogenous Adversarial Examples (EAEs) by the theory of manifold.
Our EAE adversarial training not only shortens the training time, but also enhances the robustness of the model.
- Score: 8.054201249492582
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Adversarial training is wildly considered as the most effective way to defend
against adversarial examples. However, existing adversarial training methods
consume unbearable time cost, since they need to generate adversarial examples
in the input space, which accounts for the main part of total time-consuming.
For speeding up the training process, we propose a novel adversarial training
method that does not need to generate real adversarial examples. We notice that
a clean example is closer to the decision boundary of the class with the second
largest logit component than any other class besides its own class. Thus, by
adding perturbations to logits to generate Endogenous Adversarial
Examples(EAEs) -- adversarial examples in the latent space, it can avoid
calculating gradients to speed up the training process. We further gain a deep
insight into the existence of EAEs by the theory of manifold. To guarantee the
added perturbation is within the range of constraint, we use statistical
distributions to select seed examples to craft EAEs. Extensive experiments are
conducted on CIFAR-10 and ImageNet, and the results show that compare with
state-of-the-art "Free" and "Fast" methods, our EAE adversarial training not
only shortens the training time, but also enhances the robustness of the model.
Moreover, the EAE adversarial training has little impact on the accuracy of
clean examples than the existing methods.
Related papers
- Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders [101.42201747763178]
Unlearnable examples (UEs) seek to maximize testing error by making subtle modifications to training examples that are correctly labeled.
Our work provides a novel disentanglement mechanism to build an efficient pre-training purification method.
arXiv Detail & Related papers (2024-05-02T16:49:25Z) - Fast Propagation is Better: Accelerating Single-Step Adversarial
Training via Sampling Subnetworks [69.54774045493227]
A drawback of adversarial training is the computational overhead introduced by the generation of adversarial examples.
We propose to exploit the interior building blocks of the model to improve efficiency.
Compared with previous methods, our method not only reduces the training cost but also achieves better model robustness.
arXiv Detail & Related papers (2023-10-24T01:36:20Z) - Reducing Adversarial Training Cost with Gradient Approximation [0.3916094706589679]
We propose a new and efficient adversarial training method, adversarial training with gradient approximation (GAAT) to reduce the cost of building up robust models.
Our proposed method saves up to 60% of the training time with comparable model test accuracy on datasets.
arXiv Detail & Related papers (2023-09-18T03:55:41Z) - Hard Adversarial Example Mining for Improving Robust Fairness [18.02943802341582]
Adversarial training (AT) is widely considered the state-of-the-art technique for improving the robustness of deep neural networks (DNNs) against adversarial examples (AE)
Recent studies have revealed that adversarially trained models are prone to unfairness problems, restricting their applicability.
To alleviate this problem, we propose HAM, a straightforward yet effective framework via adaptive Hard Adversarial example Mining.HAM.
arXiv Detail & Related papers (2023-08-03T15:33:24Z) - The Enemy of My Enemy is My Friend: Exploring Inverse Adversaries for
Improving Adversarial Training [72.39526433794707]
Adversarial training and its variants have been shown to be the most effective approaches to defend against adversarial examples.
We propose a novel adversarial training scheme that encourages the model to produce similar outputs for an adversarial example and its inverse adversarial'' counterpart.
Our training method achieves state-of-the-art robustness as well as natural accuracy.
arXiv Detail & Related papers (2022-11-01T15:24:26Z) - Latent Boundary-guided Adversarial Training [61.43040235982727]
Adrial training is proved to be the most effective strategy that injects adversarial examples into model training.
We propose a novel adversarial training framework called LAtent bounDary-guided aDvErsarial tRaining.
arXiv Detail & Related papers (2022-06-08T07:40:55Z) - Adaptive perturbation adversarial training: based on reinforcement
learning [9.563820241076103]
One of the shortcomings of adversarial training is that it will reduce the recognition accuracy of normal samples.
Adaptive adversarial training is proposed to alleviate this problem.
It uses marginal adversarial samples that are close to the decision boundary but does not cross the decision boundary for adversarial training.
arXiv Detail & Related papers (2021-08-30T13:49:55Z) - Multi-stage Optimization based Adversarial Training [16.295921205749934]
We propose a Multi-stage Optimization based Adversarial Training (MOAT) method that periodically trains the model on mixed benign examples.
Under similar amount of training overhead, the proposed MOAT exhibits better robustness than either single-step or multi-step adversarial training methods.
arXiv Detail & Related papers (2021-06-26T07:59:52Z) - Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.
Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks.
In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
arXiv Detail & Related papers (2020-02-14T12:36:59Z) - Efficient Adversarial Training with Transferable Adversarial Examples [58.62766224452761]
We show that there is high transferability between models from neighboring epochs in the same training process.
We propose a novel method, Adversarial Training with Transferable Adversarial Examples (ATTA) that can enhance the robustness of trained models.
arXiv Detail & Related papers (2019-12-27T03:05:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.