On the effectiveness of adversarial training against common corruptions
- URL: http://arxiv.org/abs/2103.02325v1
- Date: Wed, 3 Mar 2021 11:04:09 GMT
- Title: On the effectiveness of adversarial training against common corruptions
- Authors: Klim Kireev, Maksym Andriushchenko, Nicolas Flammarion
- Abstract summary: We show that adversarial training can serve as a strong baseline against common corruptions.
We show that our approach does not only improve the $ell_p$ adversarial training baseline but also has cumulative gains with data augmentation methods.
- Score: 29.596070201105277
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The literature on robustness towards common corruptions shows no consensus on
whether adversarial training can improve the performance in this setting.
First, we show that, when used with an appropriately selected perturbation
radius, $\ell_p$ adversarial training can serve as a strong baseline against
common corruptions. Then we explain why adversarial training performs better
than data augmentation with simple Gaussian noise which has been observed to be
a meaningful baseline on common corruptions. Related to this, we identify the
$\sigma$-overfitting phenomenon when Gaussian augmentation overfits to a
particular standard deviation used for training which has a significant
detrimental effect on common corruption accuracy. We discuss how to alleviate
this problem and then how to further enhance $\ell_p$ adversarial training by
introducing an efficient relaxation of adversarial training with learned
perceptual image patch similarity as the distance metric. Through experiments
on CIFAR-10 and ImageNet-100, we show that our approach does not only improve
the $\ell_p$ adversarial training baseline but also has cumulative gains with
data augmentation methods such as AugMix, ANT, and SIN leading to
state-of-the-art performance on common corruptions. The code of our experiments
is publicly available at https://github.com/tml-epfl/adv-training-corruptions.
Related papers
- On adversarial training and the 1 Nearest Neighbor classifier [8.248839892711478]
We analyze the adversarial robustness of the 1 Nearest Neighbor (1NN) classifier and compare its performance to adversarial training.
Our results suggest that modern adversarial training methods still fall short of the robustness of the simple 1NN classifier.
arXiv Detail & Related papers (2024-04-09T13:47:37Z) - Improved Adversarial Training Through Adaptive Instance-wise Loss
Smoothing [5.1024659285813785]
Adversarial training has been the most successful defense against such adversarial attacks.
We propose a new adversarial training method: Instance-adaptive Smoothness Enhanced Adversarial Training.
Our method achieves state-of-the-art robustness against $ell_infty$-norm constrained attacks.
arXiv Detail & Related papers (2023-03-24T15:41:40Z) - Soft Diffusion: Score Matching for General Corruptions [84.26037497404195]
We propose a new objective called Soft Score Matching that provably learns the score function for any linear corruption process.
We show that our objective learns the gradient of the likelihood under suitable regularity conditions for the family of corruption processes.
Our method achieves state-of-the-art FID score $1.85$ on CelebA-64, outperforming all previous linear diffusion models.
arXiv Detail & Related papers (2022-09-12T17:45:03Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Linear Contextual Bandits with Adversarial Corruptions [91.38793800392108]
We study the linear contextual bandit problem in the presence of adversarial corruption.
We present a variance-aware algorithm that is adaptive to the level of adversarial contamination $C$.
arXiv Detail & Related papers (2021-10-25T02:53:24Z) - Guided Interpolation for Adversarial Training [73.91493448651306]
As training progresses, the training data becomes less and less attackable, undermining the robustness enhancement.
We propose the guided framework (GIF), which employs the previous epoch's meta information to guide the data's adversarial variants.
Compared with the vanilla mixup, the GIF can provide a higher ratio of attackable data, which is beneficial to the robustness enhancement.
arXiv Detail & Related papers (2021-02-15T03:55:08Z) - Semantics-Preserving Adversarial Training [12.242659601882147]
Adversarial training is a technique that improves adversarial robustness of a deep neural network (DNN) by including adversarial examples in the training data.
We propose semantics-preserving adversarial training (SPAT) which encourages perturbation on the pixels that are shared among all classes.
Experiment results show that SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
arXiv Detail & Related papers (2020-09-23T07:42:14Z) - Overfitting in adversarially robust deep learning [86.11788847990783]
We show that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training.
We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting.
arXiv Detail & Related papers (2020-02-26T15:40:50Z) - Fast is better than free: Revisiting adversarial training [86.11788847990783]
We show that it is possible to train empirically robust models using a much weaker and cheaper adversary.
We identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail.
arXiv Detail & Related papers (2020-01-12T20:30:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.