Optimizing Information Loss Towards Robust Neural Networks

• URL: http://arxiv.org/abs/2008.03072v2
• Date: Tue, 29 Sep 2020 14:28:06 GMT
• Title: Optimizing Information Loss Towards Robust Neural Networks
• Authors: Philip Sperl and Konstantin B\"ottinger
• Abstract summary: Neural Networks (NNs) are vulnerable to adversarial examples. We present a new training approach we call textitentropic retraining. Based on an information-theoretic-inspired analysis, entropic retraining mimics the effects of adversarial training without the need of the laborious generation of adversarial examples.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Networks (NNs) are vulnerable to adversarial examples. Such inputs differ only slightly from their benign counterparts yet provoke misclassifications of the attacked NNs. The required perturbations to craft the examples are often negligible and even human imperceptible. To protect deep learning-based systems from such attacks, several countermeasures have been proposed with adversarial training still being considered the most effective. Here, NNs are iteratively retrained using adversarial examples forming a computational expensive and time consuming process often leading to a performance decrease. To overcome the downsides of adversarial training while still providing a high level of security, we present a new training approach we call \textit{entropic retraining}. Based on an information-theoretic-inspired analysis, entropic retraining mimics the effects of adversarial training without the need of the laborious generation of adversarial examples. We empirically show that entropic retraining leads to a significant increase in NNs' security and robustness while only relying on the given original data. With our prototype implementation we validate and show the effectiveness of our approach for various NN architectures and data sets.

Related papers

• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• Improved Adversarial Training Through Adaptive Instance-wise Loss Smoothing [5.1024659285813785]
  Adversarial training has been the most successful defense against such adversarial attacks. We propose a new adversarial training method: Instance-adaptive Smoothness Enhanced Adversarial Training. Our method achieves state-of-the-art robustness against $ell_infty$-norm constrained attacks.
  arXiv  Detail & Related papers  (2023-03-24T15:41:40Z)
• Adversarial training with informed data selection [53.19381941131439]
  Adrial training is the most efficient solution to defend the network against these malicious attacks. This work proposes a data selection strategy to be applied in the mini-batch training. The simulation results show that a good compromise can be obtained regarding robustness and standard accuracy.
  arXiv  Detail & Related papers  (2023-01-07T12:09:50Z)
• Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future [132.34745793391303]
  We review adversarial pretraining of self-supervised deep networks including both convolutional neural networks and vision transformers. To incorporate adversaries into pretraining models on either input or feature level, we find that existing approaches are largely categorized into two groups.
  arXiv  Detail & Related papers  (2022-10-23T13:14:06Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Latent Boundary-guided Adversarial Training [61.43040235982727]
  Adrial training is proved to be the most effective strategy that injects adversarial examples into model training. We propose a novel adversarial training framework called LAtent bounDary-guided aDvErsarial tRaining.
  arXiv  Detail & Related papers  (2022-06-08T07:40:55Z)
• Generating Adversarial Examples with Graph Neural Networks [26.74003742013481]
  We propose a novel attack based on a graph neural network (GNN) that takes advantage of the strengths of both approaches. We show that our method beats state-of-the-art adversarial attacks, including PGD-attack, MI-FGSM, and Carlini and Wagner attack. We provide a new challenging dataset specifically designed to allow for a more illustrative comparison of adversarial attacks.
  arXiv  Detail & Related papers  (2021-05-30T22:46:41Z)
• Improving adversarial robustness of deep neural networks by using semantic information [17.887586209038968]
  Adrial training is the main method for improving adversarial robustness and the first line of defense against adversarial attacks. This paper provides a new perspective on the issue of adversarial robustness, one that shifts the focus from the network as a whole to the critical part of the region close to the decision boundary corresponding to a given class. Experimental results on the MNIST and CIFAR-10 datasets show that this approach greatly improves adversarial robustness even using a very small dataset from the training data.
  arXiv  Detail & Related papers  (2020-08-18T10:23:57Z)
• On the Generalization Properties of Adversarial Training [21.79888306754263]
  This paper studies the generalization performance of a generic adversarial training algorithm. A series of numerical studies are conducted to demonstrate how the smoothness and L1 penalization help improve the adversarial robustness of models.
  arXiv  Detail & Related papers  (2020-08-15T02:32:09Z)
• Towards Achieving Adversarial Robustness by Enforcing Feature Consistency Across Bit Planes [51.31334977346847]
  We train networks to form coarse impressions based on the information in higher bit planes, and use the lower bit planes only to refine their prediction. We demonstrate that, by imposing consistency on the representations learned across differently quantized images, the adversarial robustness of networks improves significantly.
  arXiv  Detail & Related papers  (2020-04-01T09:31:10Z)
• Heat and Blur: An Effective and Fast Defense Against Adversarial Examples [2.2843885788439797]
  We propose a simple defense that combines feature visualization with input modification. We use these heatmaps as a basis for our defense, in which the adversarial effects are corrupted by massive blurring. We also provide a new evaluation metric that can capture the effects of both attacks and defenses more thoroughly and descriptively.
  arXiv  Detail & Related papers  (2020-03-17T08:11:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.