RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness

• URL: http://arxiv.org/abs/2402.06827v2
• Date: Sat, 21 Sep 2024 03:57:27 GMT
• Title: RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness
• Authors: Enyi Jiang, Gagandeep Singh,
• Abstract summary: We propose a novel training framework textbfRAMP, to boost the robustness against multiple $l_p$ perturbations. For training from scratch, textbfRAMP achieves a union accuracy of $44.6%$ and good clean accuracy of $81.2%$ on ResNet-18 against AutoAttack on CIFAR-10.
• Score: 4.188296977882316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most existing works focus on improving robustness against adversarial attacks bounded by a single $l_p$ norm using adversarial training (AT). However, these AT models' multiple-norm robustness (union accuracy) is still low, which is crucial since in the real-world an adversary is not necessarily bounded by a single norm. The tradeoffs among robustness against multiple $l_p$ perturbations and accuracy/robustness make obtaining good union and clean accuracy challenging. We design a logit pairing loss to improve the union accuracy by analyzing the tradeoffs from the lens of distribution shifts. We connect natural training (NT) with AT via gradient projection, to incorporate useful information from NT into AT, where we empirically and theoretically show it moderates the accuracy/robustness tradeoff. We propose a novel training framework \textbf{RAMP}, to boost the robustness against multiple $l_p$ perturbations. \textbf{RAMP} can be easily adapted for robust fine-tuning and full AT. For robust fine-tuning, \textbf{RAMP} obtains a union accuracy up to $53.3\%$ on CIFAR-10, and $29.1\%$ on ImageNet. For training from scratch, \textbf{RAMP} achieves a union accuracy of $44.6\%$ and good clean accuracy of $81.2\%$ on ResNet-18 against AutoAttack on CIFAR-10. Beyond multi-norm robustness \textbf{RAMP}-trained models achieve superior \textit{universal robustness}, effectively generalizing against a range of unseen adversaries and natural corruptions.

Related papers

• Towards Universal Certified Robustness with Multi-Norm Training [4.188296977882316]
  Existing certified training methods can only train models to be robust against a certain perturbation type. We propose the first multi-norm certified training framework textbfCURE, consisting of a new $l$ deterministic certified training defense. Compared with SOTA certified training, textbfCURE improves union robustness up to $22.8% on MNIST, $23.9% on CIFAR-10, and $8.0%$ on TinyImagenet.
  arXiv  Detail & Related papers  (2024-10-03T21:20:46Z)
• RUSH: Robust Contrastive Learning via Randomized Smoothing [31.717748554905015]
  In this paper, we show a surprising fact that contrastive pre-training has an interesting yet implicit connection with robustness. We design a powerful robust algorithm against adversarial attacks, RUSH, that combines the standard contrastive pre-training and randomized smoothing. Our work has an improvement of over 15% in robust accuracy and a slight improvement in standard accuracy, compared to the state-of-the-arts.
  arXiv  Detail & Related papers  (2022-07-11T18:45:14Z)
• Removing Batch Normalization Boosts Adversarial Training [83.08844497295148]
  Adversarial training (AT) defends deep neural networks against adversarial attacks. A major bottleneck is the widely used batch normalization (BN), which struggles to model the different statistics of clean and adversarial training samples in AT. Our normalizer-free robust training (NoFrost) method extends recent advances in normalizer-free networks to AT.
  arXiv  Detail & Related papers  (2022-07-04T01:39:37Z)
• Sparsity Winning Twice: Better Robust Generalization from More Efficient Training [94.92954973680914]
  We introduce two alternatives for sparse adversarial training: (i) static sparsity and (ii) dynamic sparsity. We find both methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting. Our approaches can be combined with existing regularizers, establishing new state-of-the-art results in adversarial training.
  arXiv  Detail & Related papers  (2022-02-20T15:52:08Z)
• Subspace Adversarial Training [24.47599337641455]
  We propose a new AT method, subspace adversarial training (Sub-AT), which constrains the AT in a carefully extracted subspace. In subspace, we also allow single-step AT with larger steps and larger radius, which further improves the robustness performance. Our pure single-step AT can reach over $mathbf51%$ robust accuracy against strong PGD-50 attack with radius $8/255$ on CIFAR-10.
  arXiv  Detail & Related papers  (2021-11-24T02:18:37Z)
• Robustifying $\ell_\infty$ Adversarial Training to the Union of Perturbation Models [120.71277007016708]
  We extend the capabilities of widely popular single-attack $ell_infty$ AT frameworks. Our technique, referred to as Noise Augmented Processing (SNAP), exploits a well-established byproduct of single-attack AT frameworks. SNAP prepends a given deep net with a shaped noise augmentation layer whose distribution is learned along with network parameters using any standard single-attack AT.
  arXiv  Detail & Related papers  (2021-05-31T05:18:42Z)
• Adversarial robustness against multiple $l_p$-threat models at the price of one and how to quickly fine-tune robust models to another threat model [79.05253587566197]
  Adrial training (AT) in order to achieve adversarial robustness wrt single $l_p$-threat models has been discussed extensively. In this paper we develop a simple and efficient training scheme to achieve adversarial robustness against the union of $l_p$-threat models.
  arXiv  Detail & Related papers  (2021-05-26T12:20:47Z)
• Robustness, Privacy, and Generalization of Adversarial Training [84.38148845727446]
  This paper establishes and quantifies the privacy-robustness trade-off and generalization-robustness trade-off in adversarial training. We show that adversarial training is $(varepsilon, delta)$-differentially private, where the magnitude of the differential privacy has a positive correlation with the robustified intensity. Our generalization bounds do not explicitly rely on the parameter size which would be large in deep learning.
  arXiv  Detail & Related papers  (2020-12-25T13:35:02Z)
• Toward Adversarial Robustness via Semi-supervised Robust Training [93.36310070269643]
  Adrial examples have been shown to be the severe threat to deep neural networks (DNNs) We propose a novel defense method, the robust training (RT), by jointly minimizing two separated risks ($R_stand$ and $R_rob$)
  arXiv  Detail & Related papers  (2020-03-16T02:14:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.