Rethinking Clustering for Robustness
- URL: http://arxiv.org/abs/2006.07682v3
- Date: Fri, 19 Nov 2021 18:35:02 GMT
- Title: Rethinking Clustering for Robustness
- Authors: Motasem Alfarra, Juan C. P\'erez, Adel Bibi, Ali Thabet, Pablo
Arbel\'aez, Bernard Ghanem
- Abstract summary: ClusTR is a clustering-based and adversary-free training framework to learn robust models.
textitClusTR outperforms adversarially-trained networks by up to $4%$ under strong PGD attacks.
- Score: 56.14672993686335
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper studies how encouraging semantically-aligned features during deep
neural network training can increase network robustness. Recent works observed
that Adversarial Training leads to robust models, whose learnt features appear
to correlate with human perception. Inspired by this connection from robustness
to semantics, we study the complementary connection: from semantics to
robustness. To do so, we provide a robustness certificate for distance-based
classification models (clustering-based classifiers). Moreover, we show that
this certificate is tight, and we leverage it to propose ClusTR (Clustering
Training for Robustness), a clustering-based and adversary-free training
framework to learn robust models. Interestingly, \textit{ClusTR} outperforms
adversarially-trained networks by up to $4\%$ under strong PGD attacks.
Related papers
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data [38.44734564565478]
We provide a theoretical understanding of adversarial examples and adversarial training algorithms from the perspective of feature learning theory.
We show that the adversarial training method can provably strengthen the robust feature learning and suppress the non-robust feature learning.
arXiv Detail & Related papers (2024-10-11T03:59:49Z) - Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning [49.242828934501986]
Multimodal contrastive learning has emerged as a powerful paradigm for building high-quality features.
backdoor attacks subtly embed malicious behaviors within the model during training.
We introduce an innovative token-based localized forgetting training regime.
arXiv Detail & Related papers (2024-03-24T18:33:15Z) - Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
We propose a novel doubly-robust instance reweighted adversarial framework.
Our importance weights are obtained by optimizing the KL-divergence regularized loss function.
Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
arXiv Detail & Related papers (2023-08-01T06:16:18Z) - A Theoretical Perspective on Subnetwork Contributions to Adversarial
Robustness [2.064612766965483]
This paper investigates how the adversarial robustness of a subnetwork contributes to the robustness of the entire network.
Experiments show the ability of a robust subnetwork to promote full-network robustness, and investigate the layer-wise dependencies required for this full-network robustness to be achieved.
arXiv Detail & Related papers (2023-07-07T19:16:59Z) - Understanding Robust Learning through the Lens of Representation
Similarities [37.66877172364004]
robustness to adversarial examples has emerged as a desirable property for deep neural networks (DNNs)
In this paper, we aim to understand how the properties of representations learned by robust training differ from those obtained from standard, non-robust training.
arXiv Detail & Related papers (2022-06-20T16:06:20Z) - Improving Corruption and Adversarial Robustness by Enhancing Weak
Subnets [91.9346332103637]
We propose a novel robust training method which explicitly identifies and enhances weaks during training to improve robustness.
Specifically, we develop a search algorithm to find particularly weaks and propose to explicitly strengthen them via knowledge distillation from the full network.
We show that our EWS greatly improves the robustness against corrupted images as well as the accuracy on clean data.
arXiv Detail & Related papers (2022-01-30T09:36:19Z) - An Orthogonal Classifier for Improving the Adversarial Robustness of
Neural Networks [21.13588742648554]
Recent efforts have shown that imposing certain modifications on classification layer can improve the robustness of the neural networks.
We explicitly construct a dense orthogonal weight matrix whose entries have the same magnitude, leading to a novel robust classifier.
Our method is efficient and competitive to many state-of-the-art defensive approaches.
arXiv Detail & Related papers (2021-05-19T13:12:14Z) - Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions.
We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.
We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
arXiv Detail & Related papers (2020-06-13T08:24:33Z) - HYDRA: Pruning Adversarially Robust Neural Networks [58.061681100058316]
Deep learning faces two key challenges: lack of robustness against adversarial attacks and large neural network size.
We propose to make pruning techniques aware of the robust training objective and let the training objective guide the search for which connections to prune.
We demonstrate that our approach, titled HYDRA, achieves compressed networks with state-of-the-art benign and robust accuracy, simultaneously.
arXiv Detail & Related papers (2020-02-24T19:54:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.