Enhancing Adversarial Training via Reweighting Optimization Trajectory
- URL: http://arxiv.org/abs/2306.14275v4
- Date: Sun, 4 Feb 2024 19:18:30 GMT
- Title: Enhancing Adversarial Training via Reweighting Optimization Trajectory
- Authors: Tianjin Huang, Shiwei Liu, Tianlong Chen, Meng Fang, Li Shen, Vlaod
Menkovski, Lu Yin, Yulong Pei and Mykola Pechenizkiy
- Abstract summary: A number of approaches have been proposed to address drawbacks such as extra regularization, adversarial weights, and training with more data.
We propose a new method named textbfWeighted Optimization Trajectories (WOT) that leverages the optimization trajectories of adversarial training in time.
Our results show that WOT integrates seamlessly with the existing adversarial training methods and consistently overcomes the robust overfitting issue.
- Score: 72.75558017802788
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Despite the fact that adversarial training has become the de facto method for
improving the robustness of deep neural networks, it is well-known that vanilla
adversarial training suffers from daunting robust overfitting, resulting in
unsatisfactory robust generalization. A number of approaches have been proposed
to address these drawbacks such as extra regularization, adversarial weights
perturbation, and training with more data over the last few years. However, the
robust generalization improvement is yet far from satisfactory. In this paper,
we approach this challenge with a brand new perspective -- refining historical
optimization trajectories. We propose a new method named \textbf{Weighted
Optimization Trajectories (WOT)} that leverages the optimization trajectories
of adversarial training in time. We have conducted extensive experiments to
demonstrate the effectiveness of WOT under various state-of-the-art adversarial
attacks. Our results show that WOT integrates seamlessly with the existing
adversarial training methods and consistently overcomes the robust overfitting
issue, resulting in better adversarial robustness. For example, WOT boosts the
robust accuracy of AT-PGD under AA-$L_{\infty}$ attack by 1.53\% $\sim$ 6.11\%
and meanwhile increases the clean accuracy by 0.55\%$\sim$5.47\% across SVHN,
CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets.
Related papers
- Improved Adversarial Training Through Adaptive Instance-wise Loss
Smoothing [5.1024659285813785]
Adversarial training has been the most successful defense against such adversarial attacks.
We propose a new adversarial training method: Instance-adaptive Smoothness Enhanced Adversarial Training.
Our method achieves state-of-the-art robustness against $ell_infty$-norm constrained attacks.
arXiv Detail & Related papers (2023-03-24T15:41:40Z) - WAT: Improve the Worst-class Robustness in Adversarial Training [11.872656386839436]
Adversarial training is a popular strategy to defend against adversarial attacks.
Deep Neural Networks (DNN) have been shown to be vulnerable to adversarial examples.
This paper proposes a novel framework of worst-class adversarial training.
arXiv Detail & Related papers (2023-02-08T12:54:19Z) - Consistency Regularization for Adversarial Robustness [88.65786118562005]
Adversarial training is one of the most successful methods to obtain the adversarial robustness of deep neural networks.
However, a significant generalization gap in the robustness obtained from AT has been problematic.
In this paper, we investigate data augmentation techniques to address the issue.
arXiv Detail & Related papers (2021-03-08T09:21:41Z) - A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via
Adversarial Fine-tuning [90.44219200633286]
We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy.
Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
arXiv Detail & Related papers (2020-12-25T20:50:15Z) - Self-Progressing Robust Training [146.8337017922058]
Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples.
We propose a new framework called SPROUT, self-progressing robust training.
Our results shed new light on scalable, effective and attack-independent robust training methods.
arXiv Detail & Related papers (2020-12-22T00:45:24Z) - Bag of Tricks for Adversarial Training [50.53525358778331]
Adrial training is one of the most effective strategies for promoting model robustness.
Recent benchmarks show that most of the proposed improvements on AT are less effective than simply early stopping the training procedure.
arXiv Detail & Related papers (2020-10-01T15:03:51Z) - Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning [134.15174177472807]
We introduce adversarial training into self-supervision, to provide general-purpose robust pre-trained models for the first time.
We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins.
arXiv Detail & Related papers (2020-03-28T18:28:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.