TWINS: A Fine-Tuning Framework for Improved Transferability of
Adversarial Robustness and Generalization
- URL: http://arxiv.org/abs/2303.11135v1
- Date: Mon, 20 Mar 2023 14:12:55 GMT
- Title: TWINS: A Fine-Tuning Framework for Improved Transferability of
Adversarial Robustness and Generalization
- Authors: Ziquan Liu, Yi Xu, Xiangyang Ji, Antoni B. Chan
- Abstract summary: This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks.
We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework.
TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
- Score: 89.54947228958494
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent years have seen the ever-increasing importance of pre-trained models
and their downstream training in deep learning research and applications. At
the same time, the defense for adversarial examples has been mainly
investigated in the context of training from random initialization on simple
classification tasks. To better exploit the potential of pre-trained models in
adversarial robustness, this paper focuses on the fine-tuning of an
adversarially pre-trained model in various classification tasks. Existing
research has shown that since the robust pre-trained model has already learned
a robust feature extractor, the crucial question is how to maintain the
robustness in the pre-trained model when learning the downstream task. We study
the model-based and data-based approaches for this goal and find that the two
common approaches cannot achieve the objective of improving both generalization
and adversarial robustness. Thus, we propose a novel statistics-based approach,
Two-WIng NormliSation (TWINS) fine-tuning framework, which consists of two
neural networks where one of them keeps the population means and variances of
pre-training data in the batch normalization layers. Besides the robust
information transfer, TWINS increases the effective learning rate without
hurting the training stability since the relationship between a weight norm and
its gradient norm in standard batch normalization layer is broken, resulting in
a faster escape from the sub-optimal initialization and alleviating the robust
overfitting. Finally, TWINS is shown to be effective on a wide range of image
classification datasets in terms of both generalization and robustness. Our
code is available at https://github.com/ziquanliu/CVPR2023-TWINS.
Related papers
- Adversarial Robustification via Text-to-Image Diffusion Models [56.37291240867549]
Adrial robustness has been conventionally believed as a challenging property to encode for neural networks.
We develop a scalable and model-agnostic solution to achieve adversarial robustness without using any data.
arXiv Detail & Related papers (2024-07-26T10:49:14Z) - StochCA: A Novel Approach for Exploiting Pretrained Models with Cross-Attention [2.66269503676104]
We introduce a novel fine-tuning method, called cross-attention (StochCA), specific to Transformer architectures.
This method modifies the Transformer's self-attention mechanism to selectively utilize knowledge from pretrained models during fine-tuning.
Our experimental results show the superiority of StochCA over state-of-the-art approaches in both areas.
arXiv Detail & Related papers (2024-02-25T13:53:49Z) - Activate and Reject: Towards Safe Domain Generalization under Category
Shift [71.95548187205736]
We study a practical problem of Domain Generalization under Category Shift (DGCS)
It aims to simultaneously detect unknown-class samples and classify known-class samples in the target domains.
Compared to prior DG works, we face two new challenges: 1) how to learn the concept of unknown'' during training with only source known-class samples, and 2) how to adapt the source-trained model to unseen environments.
arXiv Detail & Related papers (2023-10-07T07:53:12Z) - Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
We propose a novel doubly-robust instance reweighted adversarial framework.
Our importance weights are obtained by optimizing the KL-divergence regularized loss function.
Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
arXiv Detail & Related papers (2023-08-01T06:16:18Z) - Alleviating the Effect of Data Imbalance on Adversarial Training [26.36714114672729]
We study adversarial training on datasets that obey the long-tailed distribution.
We propose a new adversarial training framework -- Re-balancing Adversarial Training (REAT)
arXiv Detail & Related papers (2023-07-14T07:01:48Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Self-Ensemble Adversarial Training for Improved Robustness [14.244311026737666]
Adversarial training is the strongest strategy against various adversarial attacks among all sorts of defense methods.
Recent works mainly focus on developing new loss functions or regularizers, attempting to find the unique optimal point in the weight space.
We devise a simple but powerful emphSelf-Ensemble Adversarial Training (SEAT) method for yielding a robust classifier by averaging weights of history models.
arXiv Detail & Related papers (2022-03-18T01:12:18Z) - Robust Pre-Training by Adversarial Contrastive Learning [120.33706897927391]
Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness.
We improve robustness-aware self-supervised pre-training by learning representations consistent under both data augmentations and adversarial perturbations.
arXiv Detail & Related papers (2020-10-26T04:44:43Z) - Deep Ensembles for Low-Data Transfer Learning [21.578470914935938]
We study different ways of creating ensembles from pre-trained models.
We show that the nature of pre-training itself is a performant source of diversity.
We propose a practical algorithm that efficiently identifies a subset of pre-trained models for any downstream dataset.
arXiv Detail & Related papers (2020-10-14T07:59:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.