Defensive Tensorization

• URL: http://arxiv.org/abs/2110.13859v1
• Date: Tue, 26 Oct 2021 17:00:16 GMT
• Title: Defensive Tensorization
• Authors: Adrian Bulat and Jean Kossaifi and Sourav Bhattacharya and Yannis Panagakis and Timothy Hospedales and Georgios Tzimiropoulos and Nicholas D Lane and Maja Pantic
• Abstract summary: We propose tensor defensiveization, an adversarial defence technique that leverages a latent high-order factorization of the network. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We validate the versatility of our approach across domains and low-precision architectures by considering an audio task and binary networks.
• Score: 113.96183766922393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network. The layers of a network are first expressed as factorized tensor layers. Tensor dropout is then applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization.Our approach can be readily integrated with any arbitrary neural architecture and combined with techniques like adversarial training. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We validate the versatility of our approach across domains and low-precision architectures by considering an audio classification task and binary networks. In all cases, we demonstrate improved performance compared to prior works.

Related papers

• Compositional Curvature Bounds for Deep Neural Networks [7.373617024876726]
  A key challenge that threatens the widespread use of neural networks in safety-critical applications is their vulnerability to adversarial attacks. We study the second-order behavior of continuously differentiable deep neural networks, focusing on robustness against adversarial perturbations. We introduce a novel algorithm to analytically compute provable upper bounds on the second derivative of neural networks.
  arXiv  Detail & Related papers  (2024-06-07T17:50:15Z)
• Spectral regularization for adversarially-robust representation learning [32.84188052937496]
  We propose a new spectral regularizer for representation learning that encourages black-box adversarial robustness in downstream classification tasks. We show that this method is more effective in boosting test accuracy and robustness than previously-proposed methods that regularize all layers of the network.
  arXiv  Detail & Related papers  (2024-05-27T14:01:42Z)
• Robust Stochastically-Descending Unrolled Networks [85.6993263983062]
  Deep unrolling is an emerging learning-to-optimize method that unrolls a truncated iterative algorithm in the layers of a trainable neural network. We show that convergence guarantees and generalizability of the unrolled networks are still open theoretical problems. We numerically assess unrolled architectures trained under the proposed constraints in two different applications.
  arXiv  Detail & Related papers  (2023-12-25T18:51:23Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Combating Adversaries with Anti-Adversaries [118.70141983415445]
  In particular, our layer generates an input perturbation in the opposite direction of the adversarial one. We verify the effectiveness of our approach by combining our layer with both nominally and robustly trained models. Our anti-adversary layer significantly enhances model robustness while coming at no cost on clean accuracy.
  arXiv  Detail & Related papers  (2021-03-26T09:36:59Z)
• Improving Neural Network Robustness through Neighborhood Preserving Layers [0.751016548830037]
  We demonstrate a novel neural network architecture which can incorporate such layers and also can be trained efficiently. We empirically show that our designed network architecture is more robust against state-of-art gradient descent based attacks.
  arXiv  Detail & Related papers  (2021-01-28T01:26:35Z)
• Improving Adversarial Robustness by Enforcing Local and Global Compactness [19.8818435601131]
  Adversary training is the most successful method that consistently resists a wide range of attacks. We propose the Adversary Divergence Reduction Network which enforces local/global compactness and the clustering assumption. The experimental results demonstrate that augmenting adversarial training with our proposed components can further improve the robustness of the network.
  arXiv  Detail & Related papers  (2020-07-10T00:43:06Z)
• REGroup: Rank-aggregating Ensemble of Generative Classifiers for Robust Predictions [6.0162772063289784]
  Defense strategies that adopt adversarial training or random input transformations typically require retraining or fine-tuning the model to achieve reasonable performance. We find that we can learn a generative classifier by statistically characterizing the neural response of an intermediate layer to clean training samples. Our proposed approach uses a subset of the clean training data and a pre-trained model, and yet is agnostic to network architectures or the adversarial attack generation method.
  arXiv  Detail & Related papers  (2020-06-18T17:07:19Z)
• Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability [100.91186458516941]
  We consider the blackbox transfer-based targeted adversarial attack threat model in the realm of deep neural network (DNN) image classifiers. We design a flexible attack framework that allows for multi-layer perturbations and demonstrates state-of-the-art targeted transfer performance. We analyze why the proposed methods outperform existing attack strategies and show an extension of the method in the case when limited queries to the blackbox model are allowed.
  arXiv  Detail & Related papers  (2020-04-29T16:00:13Z)
• Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives [73.15276998621582]
  We propose a generic feature learning mechanism to advance CNN training with enhanced generalization ability. Partially inspired by DSN, we fork delicately designed side branches from the intermediate layers of a given neural network. Experiments on both category and instance recognition tasks demonstrate the substantial improvements of our proposed method.
  arXiv  Detail & Related papers  (2020-03-24T09:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.