Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters

• URL: http://arxiv.org/abs/2306.03430v4
• Date: Wed, 5 Jun 2024 09:46:22 GMT
• Title: Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters
• Authors: Xingxing Wei, Shiji Zhao, Bo li,
• Abstract summary: Adversarial attacks have been proven to be potential threats to Deep Neural Networks (DNNs) We propose a sample-wise dynamic network architecture named Adversarial Weight-Varied Network (AW-Net) AW-Net adaptively adjusts the network's weights based on regulation signals generated by an adversarial router.
• Score: 17.316537476091867
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial attacks have been proven to be potential threats to Deep Neural Networks (DNNs), and many methods are proposed to defend against adversarial attacks. However, while enhancing the robustness, the clean accuracy will decline to a certain extent, implying a trade-off existed between the accuracy and robustness. In this paper, to meet the trade-off problem, we theoretically explore the underlying reason for the difference of the filters' weight distribution between standard-trained and robust-trained models and then argue that this is an intrinsic property for static neural networks, thus they are difficult to fundamentally improve the accuracy and adversarial robustness at the same time. Based on this analysis, we propose a sample-wise dynamic network architecture named Adversarial Weight-Varied Network (AW-Net), which focuses on dealing with clean and adversarial examples with a "divide and rule" weight strategy. The AW-Net adaptively adjusts the network's weights based on regulation signals generated by an adversarial router, which is directly influenced by the input sample. Benefiting from the dynamic network architecture, clean and adversarial examples can be processed with different network weights, which provides the potential to enhance both accuracy and adversarial robustness. A series of experiments demonstrate that our AW-Net is architecture-friendly to handle both clean and adversarial examples and can achieve better trade-off performance than state-of-the-art robust models.

Related papers

• Beyond Pruning Criteria: The Dominant Role of Fine-Tuning and Adaptive Ratios in Neural Network Robustness [7.742297876120561]
  Deep neural networks (DNNs) excel in tasks like image recognition and natural language processing. Traditional pruning methods compromise the network's ability to withstand subtle perturbations. This paper challenges the conventional emphasis on weight importance scoring as the primary determinant of a pruned network's performance.
  arXiv  Detail & Related papers  (2024-10-19T18:35:52Z)
• From Environmental Sound Representation to Robustness of 2D CNN Models Against Adversarial Attacks [82.21746840893658]
  This paper investigates the impact of different standard environmental sound representations (spectrograms) on the recognition performance and adversarial attack robustness of a victim residual convolutional neural network. We show that while the ResNet-18 model trained on DWT spectrograms achieves a high recognition accuracy, attacking this model is relatively more costly for the adversary.
  arXiv  Detail & Related papers  (2022-04-14T15:14:08Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Non-Singular Adversarial Robustness of Neural Networks [58.731070632586594]
  Adrial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. We formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.
  arXiv  Detail & Related papers  (2021-02-23T20:59:30Z)
• Do Wider Neural Networks Really Help Adversarial Robustness? [92.8311752980399]
  We show that the model robustness is closely related to the tradeoff between natural accuracy and perturbation stability. We propose a new Width Adjusted Regularization (WAR) method that adaptively enlarges $lambda$ on wide models.
  arXiv  Detail & Related papers  (2020-10-03T04:46:17Z)
• Achieving Adversarial Robustness via Sparsity [33.11581532788394]
  We prove that the sparsity of network weights is closely associated with model robustness. We propose a novel adversarial training method called inverse weights inheritance.
  arXiv  Detail & Related papers  (2020-09-11T13:15:43Z)
• Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness [97.67477497115163]
  We use mode connectivity to study the adversarial robustness of deep neural networks. Our experiments cover various types of adversarial attacks applied to different network architectures and datasets. Our results suggest that mode connectivity offers a holistic tool and practical means for evaluating and improving adversarial robustness.
  arXiv  Detail & Related papers  (2020-04-30T19:12:50Z)
• Defense Through Diverse Directions [24.129270094757587]
  We develop a novel Bayesian neural network methodology to achieve strong adversarial robustness. We demonstrate that by encouraging the network to distribute evenly across inputs, the network becomes less susceptible to localized, brittle features. We show empirical robustness on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-24T01:22:03Z)
• Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inference [119.19779637025444]
  Deep networks were recently suggested to face the odds between accuracy (on clean natural images) and robustness (on adversarially perturbed images) This paper studies multi-exit networks associated with input-adaptive inference, showing their strong promise in achieving a "sweet point" in cooptimizing model accuracy, robustness and efficiency.
  arXiv  Detail & Related papers  (2020-02-24T00:40:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.