Towards Universal Certified Robustness with Multi-Norm Training

• URL: http://arxiv.org/abs/2410.03000v1
• Date: Thu, 3 Oct 2024 21:20:46 GMT
• Title: Towards Universal Certified Robustness with Multi-Norm Training
• Authors: Enyi Jiang, Gagandeep Singh,
• Abstract summary: Existing certified training methods can only train models to be robust against a certain perturbation type. We propose the first multi-norm certified training framework textbfCURE, consisting of a new $l$ deterministic certified training defense. Compared with SOTA certified training, textbfCURE improves union robustness up to $22.8% on MNIST, $23.9% on CIFAR-10, and $8.0%$ on TinyImagenet.
• Score: 4.188296977882316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing certified training methods can only train models to be robust against a certain perturbation type (e.g. $l_\infty$ or $l_2$). However, an $l_\infty$ certifiably robust model may not be certifiably robust against $l_2$ perturbation (and vice versa) and also has low robustness against other perturbations (e.g. geometric transformation). To this end, we propose the first multi-norm certified training framework \textbf{CURE}, consisting of a new $l_2$ deterministic certified training defense and several multi-norm certified training methods, to attain better \emph{union robustness} when training from scratch or fine-tuning a pre-trained certified model. Further, we devise bound alignment and connect natural training with certified training for better union robustness. Compared with SOTA certified training, \textbf{CURE} improves union robustness up to $22.8\%$ on MNIST, $23.9\%$ on CIFAR-10, and $8.0\%$ on TinyImagenet. Further, it leads to better generalization on a diverse set of challenging unseen geometric perturbations, up to $6.8\%$ on CIFAR-10. Overall, our contributions pave a path towards \textit{universal certified robustness}.

Related papers

• RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness [4.188296977882316]
  We propose a novel training framework textbfRAMP, to boost the robustness against multiple $l_p$ perturbations. For training from scratch, textbfRAMP achieves a union accuracy of $44.6%$ and good clean accuracy of $81.2%$ on ResNet-18 against AutoAttack on CIFAR-10.
  arXiv  Detail & Related papers  (2024-02-09T23:29:54Z)
• Raising the Bar for Certified Adversarial Robustness with Diffusion Models [9.684141378657522]
  In this work, we demonstrate that a similar approach can substantially improve deterministic certified defenses. One of our main insights is that the difference between the training and test accuracy of the original model, is a good predictor of the magnitude of the improvement. Our approach achieves state-of-the-art deterministic robustness certificates on CIFAR-10 for the $ell$ ($epsilon = 36/255$) and $ell_infty$ ($epsilon = 8/255$) threat models.
  arXiv  Detail & Related papers  (2023-05-17T17:29:10Z)
• TAPS: Connecting Certified and Adversarial Training [6.688598900034783]
  We propose TAPS, an (unsound) certified training method that combines IBP and PGD training to yield precise, although not necessarily sound, worst-case loss approximations. TAPS achieves a new state-of-the-art in many settings, e.g., reaching a certified accuracy of $22%$ on TinyImageNet.
  arXiv  Detail & Related papers  (2023-05-08T09:32:05Z)
• Sparsity Winning Twice: Better Robust Generalization from More Efficient Training [94.92954973680914]
  We introduce two alternatives for sparse adversarial training: (i) static sparsity and (ii) dynamic sparsity. We find both methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting. Our approaches can be combined with existing regularizers, establishing new state-of-the-art results in adversarial training.
  arXiv  Detail & Related papers  (2022-02-20T15:52:08Z)
• Boosting Certified $\ell_\infty$ Robustness with EMA Method and Ensemble Model [0.0]
  We introduce the EMA method to improve the training process of a $ell_infty$-norm neural network. Considering the randomness of the training algorithm, we propose an ensemble method based on trained base models with the $1$-Lipschitz property. We give the theoretical analysis of the ensemble method based on the $1$-Lipschitz property on the certified robustness, which ensures the effectiveness and stability of the algorithm.
  arXiv  Detail & Related papers  (2021-07-01T06:01:12Z)
• Robustifying $\ell_\infty$ Adversarial Training to the Union of Perturbation Models [120.71277007016708]
  We extend the capabilities of widely popular single-attack $ell_infty$ AT frameworks. Our technique, referred to as Noise Augmented Processing (SNAP), exploits a well-established byproduct of single-attack AT frameworks. SNAP prepends a given deep net with a shaped noise augmentation layer whose distribution is learned along with network parameters using any standard single-attack AT.
  arXiv  Detail & Related papers  (2021-05-31T05:18:42Z)
• Adversarial robustness against multiple $l_p$-threat models at the price of one and how to quickly fine-tune robust models to another threat model [79.05253587566197]
  Adrial training (AT) in order to achieve adversarial robustness wrt single $l_p$-threat models has been discussed extensively. In this paper we develop a simple and efficient training scheme to achieve adversarial robustness against the union of $l_p$-threat models.
  arXiv  Detail & Related papers  (2021-05-26T12:20:47Z)
• Self-Progressing Robust Training [146.8337017922058]
  Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples. We propose a new framework called SPROUT, self-progressing robust training. Our results shed new light on scalable, effective and attack-independent robust training methods.
  arXiv  Detail & Related papers  (2020-12-22T00:45:24Z)
• Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free [115.81899803240758]
  Adversarial training and its many variants substantially improve deep network robustness, yet at the cost of compromising standard accuracy. This paper asks how to quickly calibrate a trained model in-situ, to examine the achievable trade-offs between its standard and robust accuracies. Our proposed framework, Once-for-all Adversarial Training (OAT), is built on an innovative model-conditional training framework.
  arXiv  Detail & Related papers  (2020-10-22T16:06:34Z)
• Fast is better than free: Revisiting adversarial training [86.11788847990783]
  We show that it is possible to train empirically robust models using a much weaker and cheaper adversary. We identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail.
  arXiv  Detail & Related papers  (2020-01-12T20:30:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.