AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradient

• URL: http://arxiv.org/abs/2210.06888v1
• Date: Thu, 13 Oct 2022 10:31:51 GMT
• Title: AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradient
• Authors: Farzad Nikfam, Alberto Marchisio, Maurizio Martina and Muhammad Shafique
• Abstract summary: Adrial training is exploited to develop a robust Deep Neural Network (DNN) model against malicious altered data. This paper aims at accelerating the adversarial training to enable fast development of robust DNN models against adversarial attacks.
• Score: 12.118084418840152
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial training is exploited to develop a robust Deep Neural Network (DNN) model against the malicious altered data. These attacks may have catastrophic effects on DNN models but are indistinguishable for a human being. For example, an external attack can modify an image adding noises invisible for a human eye, but a DNN model misclassified the image. A key objective for developing robust DNN models is to use a learning algorithm that is fast but can also give model that is robust against different types of adversarial attacks. Especially for adversarial training, enormously long training times are needed for obtaining high accuracy under many different types of adversarial samples generated using different adversarial attack techniques. This paper aims at accelerating the adversarial training to enable fast development of robust DNN models against adversarial attacks. The general method for improving the training performance is the hyperparameters fine-tuning, where the learning rate is one of the most crucial hyperparameters. By modifying its shape (the value over time) and value during the training, we can obtain a model robust to adversarial attacks faster than standard training. First, we conduct experiments on two different datasets (CIFAR10, CIFAR100), exploring various techniques. Then, this analysis is leveraged to develop a novel fast training methodology, AccelAT, which automatically adjusts the learning rate for different epochs based on the accuracy gradient. The experiments show comparable results with the related works, and in several experiments, the adversarial training of DNNs using our AccelAT framework is conducted up to 2 times faster than the existing techniques. Thus, our findings boost the speed of adversarial training in an era in which security and performance are fundamental optimization objectives in DNN-based applications.

Related papers

• MOREL: Enhancing Adversarial Robustness through Multi-Objective Representation Learning [1.534667887016089]
  deep neural networks (DNNs) are vulnerable to slight adversarial perturbations. We show that strong feature representation learning during training can significantly enhance the original model's robustness. We propose MOREL, a multi-objective feature representation learning approach, encouraging classification models to produce similar features for inputs within the same class, despite perturbations.
  arXiv  Detail & Related papers  (2024-10-02T16:05:03Z)
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale [100.19539096465101]
  Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training. We propose a large-batch adversarial training framework implemented over multiple machines.
  arXiv  Detail & Related papers  (2022-06-13T15:39:43Z)
• Latent Boundary-guided Adversarial Training [61.43040235982727]
  Adrial training is proved to be the most effective strategy that injects adversarial examples into model training. We propose a novel adversarial training framework called LAtent bounDary-guided aDvErsarial tRaining.
  arXiv  Detail & Related papers  (2022-06-08T07:40:55Z)
• $\ell_\infty$-Robustness and Beyond: Unleashing Efficient Adversarial Training [11.241749205970253]
  We show how selecting a small subset of training data provides a more principled approach towards reducing the time complexity of robust training. Our approach speeds up adversarial training by 2-3 times, while experiencing a small reduction in the clean and robust accuracy.
  arXiv  Detail & Related papers  (2021-12-01T09:55:01Z)
• KATANA: Simple Post-Training Robustness Using Test Time Augmentations [49.28906786793494]
  A leading defense against such attacks is adversarial training, a technique in which a DNN is trained to be robust to adversarial attacks. We propose a new simple and easy-to-use technique, KATANA, for robustifying an existing pretrained DNN without modifying its weights. Our strategy achieves state-of-the-art adversarial robustness on diverse attacks with minimal compromise on the natural images' classification.
  arXiv  Detail & Related papers  (2021-09-16T19:16:00Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning [90.44219200633286]
  We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy. Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-12-25T20:50:15Z)
• Self-Progressing Robust Training [146.8337017922058]
  Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples. We propose a new framework called SPROUT, self-progressing robust training. Our results shed new light on scalable, effective and attack-independent robust training methods.
  arXiv  Detail & Related papers  (2020-12-22T00:45:24Z)
• Towards Robust Neural Networks via Orthogonal Diversity [30.77473391842894]
  A series of methods represented by the adversarial training and its variants have proven as one of the most effective techniques in enhancing the Deep Neural Networks robustness. This paper proposes a novel defense that aims at augmenting the model in order to learn features that are adaptive to diverse inputs, including adversarial examples. In this way, the proposed DIO augments the model and enhances the robustness of DNN itself as the learned features can be corrected by these mutually-orthogonal paths.
  arXiv  Detail & Related papers  (2020-10-23T06:40:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.