Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization

• URL: http://arxiv.org/abs/2003.02484v3
• Date: Mon, 27 Jul 2020 12:26:13 GMT
• Title: Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization
• Authors: Saehyung Lee, Hyungyu Lee, Sungroh Yoon
• Abstract summary: Adversarial examples cause neural networks to produce incorrect outputs with high confidence. We show that adversarial training can overshoot the optimal point in terms of robust generalization, leading to Adversarial Feature Overfitting (AFO) We propose Adversarial Vertex mixup (AVmixup) as a soft-labeled data augmentation approach for improving adversarially robust generalization.
• Score: 28.072758856453106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial examples cause neural networks to produce incorrect outputs with high confidence. Although adversarial training is one of the most effective forms of defense against adversarial examples, unfortunately, a large gap exists between test accuracy and training accuracy in adversarial training. In this paper, we identify Adversarial Feature Overfitting (AFO), which may cause poor adversarially robust generalization, and we show that adversarial training can overshoot the optimal point in terms of robust generalization, leading to AFO in our simple Gaussian model. Considering these theoretical results, we present soft labeling as a solution to the AFO problem. Furthermore, we propose Adversarial Vertex mixup (AVmixup), a soft-labeled data augmentation approach for improving adversarially robust generalization. We complement our theoretical analysis with experiments on CIFAR10, CIFAR100, SVHN, and Tiny ImageNet, and show that AVmixup significantly improves the robust generalization performance and that it reduces the trade-off between standard accuracy and adversarial robustness.

Related papers

• Adversarial Feature Alignment: Balancing Robustness and Accuracy in Deep Learning via Adversarial Training [10.099179580467737]
  Adversarial training is used to mitigate this problem by increasing robustness against adversarial attacks. This approach typically reduces a model's standard accuracy on clean, non-adversarial samples. This paper proposes a novel adversarial training method called Adversarial Feature Alignment (AFA) to address these problems.
  arXiv  Detail & Related papers  (2024-02-19T14:51:20Z)
• The Effectiveness of Random Forgetting for Robust Generalization [21.163070161951868]
  We introduce a novel learning paradigm called "Forget to Mitigate Overfitting" (FOMO) FOMO alternates between the forgetting phase, which randomly forgets a subset of weights, and the relearning phase, which emphasizes learning generalizable features. Our experiments show that FOMO alleviates robust overfitting by significantly reducing the gap between the best and last robust test accuracy.
  arXiv  Detail & Related papers  (2024-02-18T23:14:40Z)
• Enhance Diffusion to Improve Robust Generalization [39.9012723077658]
  emphAdversarial Training (AT) is one of the strongest defense mechanisms against adversarial perturbations. This paper focuses on the primary AT framework - Projected Gradient Descent Adversarial Training (PGD-AT) We propose a novel approach, emphDiffusion Enhanced Adversarial Training (DEAT), to manipulate the diffusion term to improve robust generalization with virtually no extra computational burden.
  arXiv  Detail & Related papers  (2023-06-05T06:36:18Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Sparsity Winning Twice: Better Robust Generalization from More Efficient Training [94.92954973680914]
  We introduce two alternatives for sparse adversarial training: (i) static sparsity and (ii) dynamic sparsity. We find both methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting. Our approaches can be combined with existing regularizers, establishing new state-of-the-art results in adversarial training.
  arXiv  Detail & Related papers  (2022-02-20T15:52:08Z)
• On the Convergence and Robustness of Adversarial Training [134.25999006326916]
  Adrial training with Project Gradient Decent (PGD) is amongst the most effective. We propose a textitdynamic training strategy to increase the convergence quality of the generated adversarial examples. Our theoretical and empirical results show the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2021-12-15T17:54:08Z)
• Interpolated Joint Space Adversarial Training for Robust and Generalizable Defenses [82.3052187788609]
  Adversarial training (AT) is considered to be one of the most reliable defenses against adversarial attacks. Recent works show generalization improvement with adversarial samples under novel threat models. We propose a novel threat model called Joint Space Threat Model (JSTM) Under JSTM, we develop novel adversarial attacks and defenses.
  arXiv  Detail & Related papers  (2021-12-12T21:08:14Z)
• Bridging the Gap Between Adversarial Robustness and Optimization Bias [28.56135898767349]
  Adrial robustness is an open challenge in deep learning, most often tackled using adversarial training. We show that it is possible to achieve both perfect standard accuracy and a certain degree of robustness without a trade-off. In particular, we characterize the robustness of linear convolutional models, showing that they resist attacks subject to a constraint on the Fourier-$ell_infty$ norm.
  arXiv  Detail & Related papers  (2021-02-17T16:58:04Z)
• Robust Pre-Training by Adversarial Contrastive Learning [120.33706897927391]
  Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness. We improve robustness-aware self-supervised pre-training by learning representations consistent under both data augmentations and adversarial perturbations.
  arXiv  Detail & Related papers  (2020-10-26T04:44:43Z)
• How Does Mixup Help With Robustness and Generalization? [41.58255103170875]
  We show how using Mixup in training helps model robustness and generalization. For robustness, we show that minimizing the Mixup loss corresponds to approximately minimizing an upper bound of the adversarial loss. For generalization, we prove that Mixup augmentation corresponds to a specific type of data-adaptive regularization which reduces overfitting.
  arXiv  Detail & Related papers  (2020-10-09T21:38:14Z)
• Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
  Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
  arXiv  Detail & Related papers  (2020-02-14T12:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.