TAPS: Connecting Certified and Adversarial Training
- URL: http://arxiv.org/abs/2305.04574v2
- Date: Wed, 25 Oct 2023 09:58:53 GMT
- Title: TAPS: Connecting Certified and Adversarial Training
- Authors: Yuhao Mao, Mark Niklas M\"uller, Marc Fischer, Martin Vechev
- Abstract summary: We propose TAPS, an (unsound) certified training method that combines IBP and PGD training to yield precise, although not necessarily sound, worst-case loss approximations.
TAPS achieves a new state-of-the-art in many settings, e.g., reaching a certified accuracy of $22%$ on TinyImageNet.
- Score: 6.688598900034783
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Training certifiably robust neural networks remains a notoriously hard
problem. On one side, adversarial training optimizes under-approximations of
the worst-case loss, which leads to insufficient regularization for
certification, while on the other, sound certified training methods optimize
loose over-approximations, leading to over-regularization and poor (standard)
accuracy. In this work we propose TAPS, an (unsound) certified training method
that combines IBP and PGD training to yield precise, although not necessarily
sound, worst-case loss approximations, reducing over-regularization and
increasing certified and standard accuracies. Empirically, TAPS achieves a new
state-of-the-art in many settings, e.g., reaching a certified accuracy of
$22\%$ on TinyImageNet for $\ell_\infty$-perturbations with radius
$\epsilon=1/255$. We make our implementation and networks public at
https://github.com/eth-sri/taps.
Related papers
- Towards Universal Certified Robustness with Multi-Norm Training [4.188296977882316]
Existing certified training methods can only train models to be robust against a certain perturbation type.
We propose the first multi-norm certified training framework textbfCURE, consisting of a new $l$ deterministic certified training defense.
Compared with SOTA certified training, textbfCURE improves union robustness up to $22.8% on MNIST, $23.9% on CIFAR-10, and $8.0%$ on TinyImagenet.
arXiv Detail & Related papers (2024-10-03T21:20:46Z) - Double Bubble, Toil and Trouble: Enhancing Certified Robustness through
Transitivity [27.04033198073254]
In response to subtle adversarial examples flipping classifications of neural network models, recent research has promoted certified robustness as a solution.
We show how today's "optimal" certificates can be improved by exploiting both the transitivity of certifications, and the geometry of the input space.
Our technique shows even more promising results, with a uniform $4$ percentage point increase in the achieved certified radius.
arXiv Detail & Related papers (2022-10-12T10:42:21Z) - Removing Batch Normalization Boosts Adversarial Training [83.08844497295148]
Adversarial training (AT) defends deep neural networks against adversarial attacks.
A major bottleneck is the widely used batch normalization (BN), which struggles to model the different statistics of clean and adversarial training samples in AT.
Our normalizer-free robust training (NoFrost) method extends recent advances in normalizer-free networks to AT.
arXiv Detail & Related papers (2022-07-04T01:39:37Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Smooth-Reduce: Leveraging Patches for Improved Certified Robustness [100.28947222215463]
We propose a training-free, modified smoothing approach, Smooth-Reduce.
Our algorithm classifies overlapping patches extracted from an input image, and aggregates the predicted logits to certify a larger radius around the input.
We provide theoretical guarantees for such certificates, and empirically show significant improvements over other randomized smoothing methods.
arXiv Detail & Related papers (2022-05-12T15:26:20Z) - Sparsity Winning Twice: Better Robust Generalization from More Efficient
Training [94.92954973680914]
We introduce two alternatives for sparse adversarial training: (i) static sparsity and (ii) dynamic sparsity.
We find both methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting.
Our approaches can be combined with existing regularizers, establishing new state-of-the-art results in adversarial training.
arXiv Detail & Related papers (2022-02-20T15:52:08Z) - Boosting Certified $\ell_\infty$ Robustness with EMA Method and Ensemble
Model [0.0]
We introduce the EMA method to improve the training process of a $ell_infty$-norm neural network.
Considering the randomness of the training algorithm, we propose an ensemble method based on trained base models with the $1$-Lipschitz property.
We give the theoretical analysis of the ensemble method based on the $1$-Lipschitz property on the certified robustness, which ensures the effectiveness and stability of the algorithm.
arXiv Detail & Related papers (2021-07-01T06:01:12Z) - Fast Certified Robust Training via Better Initialization and Shorter
Warmup [95.81628508228623]
We propose a new IBP and principled regularizers during the warmup stage to stabilize certified bounds.
We find that batch normalization (BN) is a crucial architectural element to build best-performing networks for certified training.
arXiv Detail & Related papers (2021-03-31T17:58:58Z) - Activation Density driven Energy-Efficient Pruning in Training [2.222917681321253]
We propose a novel pruning method that prunes a network real-time during training.
We obtain exceedingly sparse networks with accuracy comparable to the baseline network.
arXiv Detail & Related papers (2020-02-07T18:34:31Z) - Fast is better than free: Revisiting adversarial training [86.11788847990783]
We show that it is possible to train empirically robust models using a much weaker and cheaper adversary.
We identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail.
arXiv Detail & Related papers (2020-01-12T20:30:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.