THAT: Two Head Adversarial Training for Improving Robustness at Scale
- URL: http://arxiv.org/abs/2103.13612v1
- Date: Thu, 25 Mar 2021 05:32:38 GMT
- Title: THAT: Two Head Adversarial Training for Improving Robustness at Scale
- Authors: Zuxuan Wu, Tom Goldstein, Larry S. Davis, Ser-Nam Lim
- Abstract summary: We propose Two Head Adversarial Training (THAT), a two-stream adversarial learning network that is designed to handle the large-scale many-class ImageNet dataset.
The proposed method trains a network with two heads and two loss functions; one to minimize feature-space domain shift between natural and adversarial images, and one to promote high classification accuracy.
- Score: 126.06873298511425
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Many variants of adversarial training have been proposed, with most research
focusing on problems with relatively few classes. In this paper, we propose Two
Head Adversarial Training (THAT), a two-stream adversarial learning network
that is designed to handle the large-scale many-class ImageNet dataset. The
proposed method trains a network with two heads and two loss functions; one to
minimize feature-space domain shift between natural and adversarial images, and
one to promote high classification accuracy. This combination delivers a
hardened network that achieves state of the art robust accuracy while
maintaining high natural accuracy on ImageNet. Through extensive experiments,
we demonstrate that the proposed framework outperforms alternative methods
under both standard and "free" adversarial training settings.
Related papers
- A Comprehensive Study on Robustness of Image Classification Models:
Benchmarking and Rethinking [54.89987482509155]
robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts.
We establish a comprehensive benchmark robustness called textbfARES-Bench on the image classification task.
By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness.
arXiv Detail & Related papers (2023-02-28T04:26:20Z) - Enhancing Adversarial Training with Feature Separability [52.39305978984573]
We introduce a new concept of adversarial training graph (ATG) with which the proposed adversarial training with feature separability (ATFS) enables to boost the intra-class feature similarity and increase inter-class feature variance.
Through comprehensive experiments, we demonstrate that the proposed ATFS framework significantly improves both clean and robust performance.
arXiv Detail & Related papers (2022-05-02T04:04:23Z) - Learning Representations Robust to Group Shifts and Adversarial Examples [18.742222861886148]
We propose an algorithm that combines adversarial training and group distribution robust optimization to improve representation learning.
Experiments on three image benchmark datasets illustrate that the proposed method achieves superior results on robust metrics without sacrificing much of the standard measures.
arXiv Detail & Related papers (2022-02-18T22:06:25Z) - Dual Head Adversarial Training [31.538325500032]
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples/attacks.
Recent studies have shown that there exists an inherent tradeoff between accuracy and robustness in adversarially-trained DNNs.
We propose a novel technique Dual Head Adversarial Training (DH-AT) to further improve the robustness of existing adversarial training methods.
arXiv Detail & Related papers (2021-04-21T06:31:33Z) - Learning Neural Network Subspaces [74.44457651546728]
Recent observations have advanced our understanding of the neural network optimization landscape.
With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks.
With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks.
arXiv Detail & Related papers (2021-02-20T23:26:58Z) - Deep Artifact-Free Residual Network for Single Image Super-Resolution [0.2399911126932526]
We propose Deep Artifact-Free Residual (DAFR) network which uses the merits of both residual learning and usage of ground-truth image as target.
Our framework uses a deep model to extract the high-frequency information which is necessary for high-quality image reconstruction.
Our experimental results show that the proposed method achieves better quantitative and qualitative image quality compared to the existing methods.
arXiv Detail & Related papers (2020-09-25T20:53:55Z) - Stylized Adversarial Defense [105.88250594033053]
adversarial training creates perturbation patterns and includes them in the training set to robustify the model.
We propose to exploit additional information from the feature space to craft stronger adversaries.
Our adversarial training approach demonstrates strong robustness compared to state-of-the-art defenses.
arXiv Detail & Related papers (2020-07-29T08:38:10Z) - Class-Aware Domain Adaptation for Improving Adversarial Robustness [27.24720754239852]
adversarial training has been proposed to train networks by injecting adversarial examples into the training data.
We propose a novel Class-Aware Domain Adaptation (CADA) method for adversarial defense without directly applying adversarial training.
arXiv Detail & Related papers (2020-05-10T03:45:19Z) - Towards Achieving Adversarial Robustness by Enforcing Feature
Consistency Across Bit Planes [51.31334977346847]
We train networks to form coarse impressions based on the information in higher bit planes, and use the lower bit planes only to refine their prediction.
We demonstrate that, by imposing consistency on the representations learned across differently quantized images, the adversarial robustness of networks improves significantly.
arXiv Detail & Related papers (2020-04-01T09:31:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.