Wiggling Weights to Improve the Robustness of Classifiers

• URL: http://arxiv.org/abs/2111.09779v1
• Date: Thu, 18 Nov 2021 16:20:36 GMT
• Title: Wiggling Weights to Improve the Robustness of Classifiers
• Authors: Sadaf Gulshad, Ivan Sosnovik, Arnold Smeulders
• Abstract summary: We show that wiggling the weights consistently improves classification. We conclude that wiggled transform-augmented networks acquire good robustness even for perturbations not seen during training.
• Score: 2.1485350418225244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robustness against unwanted perturbations is an important aspect of deploying neural network classifiers in the real world. Common natural perturbations include noise, saturation, occlusion, viewpoint changes, and blur deformations. All of them can be modelled by the newly proposed transform-augmented convolutional networks. While many approaches for robustness train the network by providing augmented data to the network, we aim to integrate perturbations in the network architecture to achieve improved and more general robustness. To demonstrate that wiggling the weights consistently improves classification, we choose a standard network and modify it to a transform-augmented network. On perturbed CIFAR-10 images, the modified network delivers a better performance than the original network. For the much smaller STL-10 dataset, in addition to delivering better general robustness, wiggling even improves the classification of unperturbed, clean images substantially. We conclude that wiggled transform-augmented networks acquire good robustness even for perturbations not seen during training.

Related papers

• Beyond Pruning Criteria: The Dominant Role of Fine-Tuning and Adaptive Ratios in Neural Network Robustness [7.742297876120561]
  Deep neural networks (DNNs) excel in tasks like image recognition and natural language processing. Traditional pruning methods compromise the network's ability to withstand subtle perturbations. This paper challenges the conventional emphasis on weight importance scoring as the primary determinant of a pruned network's performance.
  arXiv  Detail & Related papers  (2024-10-19T18:35:52Z)
• An Enhanced Encoder-Decoder Network Architecture for Reducing Information Loss in Image Semantic Segmentation [6.596361762662328]
  We introduce an innovative encoder-decoder network structure enhanced with residual connections. Our approach employs a multi-residual connection strategy designed to preserve the intricate details across various image scales more effectively. To enhance the convergence rate of network training and mitigate sample imbalance issues, we have devised a modified cross-entropy loss function.
  arXiv  Detail & Related papers  (2024-05-26T05:15:53Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• Improving Corruption and Adversarial Robustness by Enhancing Weak Subnets [91.9346332103637]
  We propose a novel robust training method which explicitly identifies and enhances weaks during training to improve robustness. Specifically, we develop a search algorithm to find particularly weaks and propose to explicitly strengthen them via knowledge distillation from the full network. We show that our EWS greatly improves the robustness against corrupted images as well as the accuracy on clean data.
  arXiv  Detail & Related papers  (2022-01-30T09:36:19Z)
• Evolving Architectures with Gradient Misalignment toward Low Adversarial Transferability [4.415977307120616]
  We propose an architecture searching framework that employs neuroevolution to evolve network architectures. Our experiments show that the proposed framework successfully discovers architectures that reduce transferability from four standard networks. In addition, the evolved networks trained with gradient misalignment exhibit significantly lower transferability compared to standard networks trained with gradient misalignment.
  arXiv  Detail & Related papers  (2021-09-13T12:41:53Z)
• Extreme Value Preserving Networks [65.2037926048262]
  Recent evidence shows that convolutional neural networks (CNNs) are biased towards textures so that CNNs are non-robust to adversarial perturbations over textures. This paper aims to leverage good properties of SIFT to renovate CNN architectures towards better accuracy and robustness.
  arXiv  Detail & Related papers  (2020-11-17T02:06:52Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)
• Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations [52.493315075385325]
  We show that a family of regularizers, including weight decay, is ineffective at penalizing the intrinsic norms of weights for networks with homogeneous activation functions. We propose an improved regularizer that is invariant to weight scale shifting and thus effectively constrains the intrinsic norm of a neural network.
  arXiv  Detail & Related papers  (2020-08-07T02:55:28Z)
• On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
  Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time. We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
  arXiv  Detail & Related papers  (2020-07-16T18:39:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.