Adversarial Concurrent Training: Optimizing Robustness and Accuracy
Trade-off of Deep Neural Networks
- URL: http://arxiv.org/abs/2008.07015v2
- Date: Tue, 18 Aug 2020 18:31:40 GMT
- Title: Adversarial Concurrent Training: Optimizing Robustness and Accuracy
Trade-off of Deep Neural Networks
- Authors: Elahe Arani, Fahad Sarfraz and Bahram Zonooz
- Abstract summary: We propose Adversarial Concurrent Training (ACT) to train a robust model in conjunction with a natural model in a minimax game.
ACT achieves 68.20% standard accuracy and 44.29% robustness accuracy under a 100-iteration untargeted attack.
- Score: 13.041607703862724
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Adversarial training has been proven to be an effective technique for
improving the adversarial robustness of models. However, there seems to be an
inherent trade-off between optimizing the model for accuracy and robustness. To
this end, we propose Adversarial Concurrent Training (ACT), which employs
adversarial training in a collaborative learning framework whereby we train a
robust model in conjunction with a natural model in a minimax game. ACT
encourages the two models to align their feature space by using the
task-specific decision boundaries and explore the input space more broadly.
Furthermore, the natural model acts as a regularizer, enforcing priors on
features that the robust model should learn. Our analyses on the behavior of
the models show that ACT leads to a robust model with lower model complexity,
higher information compression in the learned representations, and high
posterior entropy solutions indicative of convergence to a flatter minima. We
demonstrate the effectiveness of the proposed approach across different
datasets and network architectures. On ImageNet, ACT achieves 68.20% standard
accuracy and 44.29% robustness accuracy under a 100-iteration untargeted
attack, improving upon the standard adversarial training method's 65.70%
standard accuracy and 42.36% robustness.
Related papers
- Robustness-Congruent Adversarial Training for Secure Machine Learning
Model Updates [13.911586916369108]
We show that misclassifications in machine-learning models can affect robustness to adversarial examples.
We propose a technique, named robustness-congruent adversarial training, to address this issue.
We show that our algorithm and, more generally, learning with non-regression constraints, provides a theoretically-grounded framework to train consistent estimators.
arXiv Detail & Related papers (2024-02-27T10:37:13Z) - Learn from the Past: A Proxy Guided Adversarial Defense Framework with
Self Distillation Regularization [53.04697800214848]
Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models.
AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting.
We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
arXiv Detail & Related papers (2023-10-19T13:13:41Z) - Adaptive Certified Training: Towards Better Accuracy-Robustness
Tradeoffs [17.46692880231195]
We propose a novel certified training method based on a key insight that training with adaptive certified radii helps to improve the accuracy and robustness of the model.
We demonstrate the effectiveness of the proposed method on MNIST, CIFAR-10, and TinyImageNet datasets.
arXiv Detail & Related papers (2023-07-24T18:59:46Z) - Adaptive Modeling Against Adversarial Attacks [1.90365714903665]
Adversarial training, the process of training a deep learning model with adversarial data, is one of the most successful adversarial defense methods for deep learning models.
We have found that the robustness to white-box attack of an adversarially trained model can be further improved if we fine tune this model in inference stage to adapt to the adversarial input.
arXiv Detail & Related papers (2021-12-23T09:52:30Z) - Mutual Adversarial Training: Learning together is better than going
alone [82.78852509965547]
We study how interactions among models affect robustness via knowledge distillation.
We propose mutual adversarial training (MAT) in which multiple models are trained together.
MAT can effectively improve model robustness and outperform state-of-the-art methods under white-box attacks.
arXiv Detail & Related papers (2021-12-09T15:59:42Z) - No One Representation to Rule Them All: Overlapping Features of Training
Methods [12.58238785151714]
High-performing models tend to make similar predictions regardless of training methodology.
Recent work has made very different training techniques, such as large-scale contrastive learning, yield competitively-high accuracy.
We show these models specialize in generalization of the data, leading to higher ensemble performance.
arXiv Detail & Related papers (2021-10-20T21:29:49Z) - A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via
Adversarial Fine-tuning [90.44219200633286]
We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy.
Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
arXiv Detail & Related papers (2020-12-25T20:50:15Z) - Firearm Detection via Convolutional Neural Networks: Comparing a
Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents.
One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis.
We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
arXiv Detail & Related papers (2020-12-17T15:19:29Z) - Voting based ensemble improves robustness of defensive models [82.70303474487105]
We study whether it is possible to create an ensemble to further improve robustness.
By ensembling several state-of-the-art pre-trained defense models, our method can achieve a 59.8% robust accuracy.
arXiv Detail & Related papers (2020-11-28T00:08:45Z) - Improved Adversarial Training via Learned Optimizer [101.38877975769198]
We propose a framework to improve the robustness of adversarial training models.
By co-training's parameters model's weights, the proposed framework consistently improves robustness and steps adaptively for update directions.
arXiv Detail & Related papers (2020-04-25T20:15:53Z) - Revisiting Ensembles in an Adversarial Context: Improving Natural
Accuracy [5.482532589225552]
There is still a significant gap in natural accuracy between robust and non-robust models.
We consider a number of ensemble methods designed to mitigate this performance difference.
We consider two schemes, one that combines predictions from several randomly robust models, and the other that fuses features from robust and standard models.
arXiv Detail & Related papers (2020-02-26T15:45:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.