Consistency Regularization for Adversarial Robustness

• URL: http://arxiv.org/abs/2103.04623v1
• Date: Mon, 8 Mar 2021 09:21:41 GMT
• Title: Consistency Regularization for Adversarial Robustness
• Authors: Jihoon Tack, Sihyun Yu, Jongheon Jeong, Minseon Kim, Sung Ju Hwang, Jinwoo Shin
• Abstract summary: Adversarial training is one of the most successful methods to obtain the adversarial robustness of deep neural networks. However, a significant generalization gap in the robustness obtained from AT has been problematic. In this paper, we investigate data augmentation techniques to address the issue.
• Score: 88.65786118562005
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial training (AT) is currently one of the most successful methods to obtain the adversarial robustness of deep neural networks. However, a significant generalization gap in the robustness obtained from AT has been problematic, making practitioners to consider a bag of tricks for a successful training, e.g., early stopping. In this paper, we investigate data augmentation (DA) techniques to address the issue. In contrast to the previous reports in the literature that DA is not effective for regularizing AT, we discover that DA can mitigate overfitting in AT surprisingly well, but they should be chosen deliberately. To utilize the effect of DA further, we propose a simple yet effective auxiliary 'consistency' regularization loss to optimize, which forces predictive distributions after attacking from two different augmentations to be similar to each other. Our experimental results demonstrate that our simple regularization scheme is applicable for a wide range of AT methods, showing consistent yet significant improvements in the test robust accuracy. More remarkably, we also show that our method could significantly help the model to generalize its robustness against unseen adversaries, e.g., other types or larger perturbations compared to those used during training. Code is available at https://github.com/alinlab/consistency-adversarial.

Related papers

• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• Enhancing Adversarial Training via Reweighting Optimization Trajectory [72.75558017802788]
  A number of approaches have been proposed to address drawbacks such as extra regularization, adversarial weights, and training with more data. We propose a new method named textbfWeighted Optimization Trajectories (WOT) that leverages the optimization trajectories of adversarial training in time. Our results show that WOT integrates seamlessly with the existing adversarial training methods and consistently overcomes the robust overfitting issue.
  arXiv  Detail & Related papers  (2023-06-25T15:53:31Z)
• AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation [6.625868719336385]
  Adversarial training (AT) is an effective defense against adversarial examples. Data augmentation (DA) was shown to be effective in mitigating robust overfitting if appropriately designed and optimized for AT. This work proposes a new method to automatically learn online, instance-wise, DA policies to improve robust generalization for AT.
  arXiv  Detail & Related papers  (2023-06-12T15:54:52Z)
• Enhance Diffusion to Improve Robust Generalization [39.9012723077658]
  emphAdversarial Training (AT) is one of the strongest defense mechanisms against adversarial perturbations. This paper focuses on the primary AT framework - Projected Gradient Descent Adversarial Training (PGD-AT) We propose a novel approach, emphDiffusion Enhanced Adversarial Training (DEAT), to manipulate the diffusion term to improve robust generalization with virtually no extra computational burden.
  arXiv  Detail & Related papers  (2023-06-05T06:36:18Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Alleviating Robust Overfitting of Adversarial Training With Consistency Regularization [9.686724616328874]
  Adversarial training (AT) has proven to be one of the most effective ways to defend Deep Neural Networks (DNNs) against adversarial attacks. robustness will drop sharply at a certain stage, always exists during AT. consistency regularization, a popular technique in semi-supervised learning, has a similar goal as AT and can be used to alleviate robust overfitting.
  arXiv  Detail & Related papers  (2022-05-24T03:18:43Z)
• Exploring Adversarially Robust Training for Unsupervised Domain Adaptation [71.94264837503135]
  Unsupervised Domain Adaptation (UDA) methods aim to transfer knowledge from a labeled source domain to an unlabeled target domain. This paper explores how to enhance the unlabeled data robustness via AT while learning domain-invariant features for UDA. We propose a novel Adversarially Robust Training method for UDA accordingly, referred to as ARTUDA.
  arXiv  Detail & Related papers  (2022-02-18T17:05:19Z)
• Bag of Tricks for Adversarial Training [50.53525358778331]
  Adrial training is one of the most effective strategies for promoting model robustness. Recent benchmarks show that most of the proposed improvements on AT are less effective than simply early stopping the training procedure.
  arXiv  Detail & Related papers  (2020-10-01T15:03:51Z)
• Second Order Optimization for Adversarial Robustness and Interpretability [6.700873164609009]
  We propose a novel regularizer which incorporates first and second order information via a quadratic approximation to the adversarial loss. It is shown that using only a single iteration in our regularizer achieves stronger robustness than prior gradient and curvature regularization schemes. It retains the interesting facet of AT that networks learn features which are well-aligned with human perception.
  arXiv  Detail & Related papers  (2020-09-10T15:05:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.