Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network

• URL: http://arxiv.org/abs/2204.02738v1
• Date: Wed, 6 Apr 2022 11:28:06 GMT
• Title: Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network
• Authors: Byung-Kwan Lee, Junho Kim, Yong Man Ro
• Abstract summary: Adversarial examples provoke weak reliability and potential security issues in deep neural networks. We propose a novel adversarial pruning method, Masking Adversarial Damage (MAD) that employs second-order information of adversarial loss. We show that MAD effectively prunes adversarially trained networks without loosing adversarial robustness and shows better performance than previous adversarial pruning methods.
• Score: 33.18197518590706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial examples provoke weak reliability and potential security issues in deep neural networks. Although adversarial training has been widely studied to improve adversarial robustness, it works in an over-parameterized regime and requires high computations and large memory budgets. To bridge adversarial robustness and model compression, we propose a novel adversarial pruning method, Masking Adversarial Damage (MAD) that employs second-order information of adversarial loss. By using it, we can accurately estimate adversarial saliency for model parameters and determine which parameters can be pruned without weakening adversarial robustness. Furthermore, we reveal that model parameters of initial layer are highly sensitive to the adversarial examples and show that compressed feature representation retains semantic information for the target objects. Through extensive experiments on three public datasets, we demonstrate that MAD effectively prunes adversarially trained networks without loosing adversarial robustness and shows better performance than previous adversarial pruning methods.

Related papers

• Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis [25.993502776271022]
  Having a large parameter space is considered one of the main suspects of the neural networks' vulnerability to adversarial example. Previous research has demonstrated that depending on the considered model, the algorithm employed to generate adversarial examples may not function properly.
  arXiv  Detail & Related papers  (2024-06-14T14:47:06Z)
• Revisiting DeepFool: generalization and improvement [17.714671419826715]
  We introduce a new family of adversarial attacks that strike a balance between effectiveness and computational efficiency. Our proposed attacks are also suitable for evaluating the robustness of large models.
  arXiv  Detail & Related papers  (2023-03-22T11:49:35Z)
• Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning [80.21709045433096]
  A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample. We use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
  arXiv  Detail & Related papers  (2022-11-04T13:54:02Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• Modelling Adversarial Noise for Adversarial Defense [96.56200586800219]
  adversarial defenses typically focus on exploiting adversarial examples to remove adversarial noise or train an adversarially robust target model. Motivated by that the relationship between adversarial data and natural data can help infer clean data from adversarial data to obtain the final correct prediction. We study to model adversarial noise to learn the transition relationship in the label space for using adversarial labels to improve adversarial accuracy.
  arXiv  Detail & Related papers  (2021-09-21T01:13:26Z)
• Pruning in the Face of Adversaries [0.0]
  We evaluate the impact of neural network pruning on the adversarial robustness against L-0, L-2 and L-infinity attacks. Our results confirm that neural network pruning and adversarial robustness are not mutually exclusive. We extend our analysis to situations that incorporate additional assumptions on the adversarial scenario and show that depending on the situation, different strategies are optimal.
  arXiv  Detail & Related papers  (2021-08-19T09:06:16Z)
• Residual Error: a New Performance Measure for Adversarial Robustness [85.0371352689919]
  A major challenge that limits the wide-spread adoption of deep learning has been their fragility to adversarial attacks. This study presents the concept of residual error, a new performance measure for assessing the adversarial robustness of a deep neural network. Experimental results using the case of image classification demonstrate the effectiveness and efficacy of the proposed residual error metric.
  arXiv  Detail & Related papers  (2021-06-18T16:34:23Z)
• Stylized Adversarial Defense [105.88250594033053]
  adversarial training creates perturbation patterns and includes them in the training set to robustify the model. We propose to exploit additional information from the feature space to craft stronger adversaries. Our adversarial training approach demonstrates strong robustness compared to state-of-the-art defenses.
  arXiv  Detail & Related papers  (2020-07-29T08:38:10Z)
• Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness [97.67477497115163]
  We use mode connectivity to study the adversarial robustness of deep neural networks. Our experiments cover various types of adversarial attacks applied to different network architectures and datasets. Our results suggest that mode connectivity offers a holistic tool and practical means for evaluating and improving adversarial robustness.
  arXiv  Detail & Related papers  (2020-04-30T19:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.