Towards Natural Robustness Against Adversarial Examples

• URL: http://arxiv.org/abs/2012.02452v1
• Date: Fri, 4 Dec 2020 08:12:38 GMT
• Title: Towards Natural Robustness Against Adversarial Examples
• Authors: Haoyu Chu, Shikui Wei, Yao Zhao
• Abstract summary: We show that a new family of deep neural networks called Neural ODEs holds a weaker upper bound. This weaker upper bound prevents the amount of change in the result from being too large. We show that the natural robustness of Neural ODEs is even better than the robustness of neural networks that are trained with adversarial training methods.
• Score: 35.5696648642793
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies have shown that deep neural networks are vulnerable to adversarial examples, but most of the methods proposed to defense adversarial examples cannot solve this problem fundamentally. In this paper, we theoretically prove that there is an upper bound for neural networks with identity mappings to constrain the error caused by adversarial noises. However, in actual computations, this kind of neural network no longer holds any upper bound and is therefore susceptible to adversarial examples. Following similar procedures, we explain why adversarial examples can fool other deep neural networks with skip connections. Furthermore, we demonstrate that a new family of deep neural networks called Neural ODEs (Chen et al., 2018) holds a weaker upper bound. This weaker upper bound prevents the amount of change in the result from being too large. Thus, Neural ODEs have natural robustness against adversarial examples. We evaluate the performance of Neural ODEs compared with ResNet under three white-box adversarial attacks (FGSM, PGD, DI2-FGSM) and one black-box adversarial attack (Boundary Attack). Finally, we show that the natural robustness of Neural ODEs is even better than the robustness of neural networks that are trained with adversarial training methods, such as TRADES and YOPO.

Related papers

• Towards unlocking the mystery of adversarial fragility of neural networks [6.589200529058999]
  We look at the smallest magnitude of possible additive perturbations that can change the output of a classification algorithm. We provide a matrix-theoretic explanation of the adversarial fragility of deep neural network for classification.
  arXiv  Detail & Related papers  (2024-06-23T19:37:13Z)
• Finite Gaussian Neurons: Defending against adversarial attacks by making neural networks say "I don't know" [0.0]
  I introduce the Finite Gaussian Neuron (FGN), a novel neuron architecture for artificial neural networks. My works aims to: - easily convert existing models to FGN architecture, - while preserving the existing model's behavior on real data, - and offering resistance against adversarial attacks.
  arXiv  Detail & Related papers  (2023-06-13T14:17:25Z)
• Benign Overfitting for Two-layer ReLU Convolutional Neural Networks [60.19739010031304]
  We establish algorithm-dependent risk bounds for learning two-layer ReLU convolutional neural networks with label-flipping noise. We show that, under mild conditions, the neural network trained by gradient descent can achieve near-zero training loss and Bayes optimal test risk.
  arXiv  Detail & Related papers  (2023-03-07T18:59:38Z)
• Searching for the Essence of Adversarial Perturbations [73.96215665913797]
  We show that adversarial perturbations contain human-recognizable information, which is the key conspirator responsible for a neural network's erroneous prediction. This concept of human-recognizable information allows us to explain key features related to adversarial perturbations.
  arXiv  Detail & Related papers  (2022-05-30T18:04:57Z)
• Adversarial Robustness in Deep Learning: Attacks on Fragile Neurons [0.6899744489931016]
  We identify fragile and robust neurons of deep learning architectures using nodal dropouts of the first convolutional layer. We correlate these neurons with the distribution of adversarial attacks on the network.
  arXiv  Detail & Related papers  (2022-01-31T14:34:07Z)
• Thundernna: a white box adversarial attack [0.0]
  We develop a first-order method to attack the neural network. Compared with other first-order attacks, our method has a much higher success rate.
  arXiv  Detail & Related papers  (2021-11-24T07:06:21Z)
• On the Adversarial Robustness of Quantized Neural Networks [2.0625936401496237]
  It is unclear how model compression techniques may affect the robustness of AI algorithms against adversarial attacks. This paper explores the effect of quantization, one of the most common compression techniques, on the adversarial robustness of neural networks.
  arXiv  Detail & Related papers  (2021-05-01T11:46:35Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting [135.0863818867184]
  artificial neural variability (ANV) helps artificial neural networks learn some advantages from natural'' neural networks. ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. It can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.
  arXiv  Detail & Related papers  (2020-11-12T06:06:33Z)
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models [58.47179090632039]
  A key challenge in adversarial robustness is the lack of a precise mathematical characterization of human perception. Under the neural perceptual threat model, we develop novel perceptual adversarial attacks and defenses. Because the NPTM is very broad, we find that Perceptual Adrial Training (PAT) against a perceptual attack gives robustness against many other types of adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-22T22:40:46Z)
• Feature Purification: How Adversarial Training Performs Robust Deep Learning [66.05472746340142]
  We show a principle that we call Feature Purification, where we show one of the causes of the existence of adversarial examples is the accumulation of certain small dense mixtures in the hidden weights during the training process of a neural network. We present both experiments on the CIFAR-10 dataset to illustrate this principle, and a theoretical result proving that for certain natural classification tasks, training a two-layer neural network with ReLU activation using randomly gradient descent indeed this principle.
  arXiv  Detail & Related papers  (2020-05-20T16:56:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.