ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients

• URL: http://arxiv.org/abs/2108.04217v1
• Date: Tue, 6 Jul 2021 12:03:36 GMT
• Title: ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients
• Authors: Alessandro Cappelli, Julien Launay, Laurent Meunier, Ruben Ohana and Iacopo Poli
• Abstract summary: We introduce ROPUST, a simple and efficient method to leverage robust pre-trained models and increase their robustness. We test our method on nine different models against four attacks in RobustBench, consistently improving over state-of-the-art performance. We show that even with state-of-the-art phase retrieval techniques, ROPUST remains an effective defense.
• Score: 65.52888259961803
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robustness to adversarial attacks is typically obtained through expensive adversarial training with Projected Gradient Descent. Here we introduce ROPUST, a remarkably simple and efficient method to leverage robust pre-trained models and further increase their robustness, at no cost in natural accuracy. Our technique relies on the use of an Optical Processing Unit (OPU), a photonic co-processor, and a fine-tuning step performed with Direct Feedback Alignment, a synthetic gradient training scheme. We test our method on nine different models against four attacks in RobustBench, consistently improving over state-of-the-art performance. We perform an ablation study on the single components of our defense, showing that robustness arises from parameter obfuscation and the alternative training method. We also introduce phase retrieval attacks, specifically designed to increase the threat level of attackers against our own defense. We show that even with state-of-the-art phase retrieval techniques, ROPUST remains an effective defense.

Related papers

• Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization [29.09894840783714]
  We propose a novel pipeline to acquire the robust purifier model, named Adversarial Training on Purification (AToP) To evaluate our method in an efficient and scalable way, we conduct extensive experiments on CIFAR-10, CIFAR-100, and ImageNette.
  arXiv  Detail & Related papers  (2024-01-29T17:56:42Z)
• How Robust Are Energy-Based Models Trained With Equilibrium Propagation? [4.374837991804085]
  Adrial training is the current state-of-the-art defense against adversarial attacks. It lowers the model's accuracy on clean inputs, is computationally expensive, and offers less robustness to natural noise. In contrast, energy-based models (EBMs) incorporate feedback connections from each layer to the previous layer, yielding a recurrent, deep-attractor architecture.
  arXiv  Detail & Related papers  (2024-01-21T16:55:40Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training [106.34722726264522]
  A range of adversarial defense techniques have been proposed to mitigate the interference of adversarial noise. Pre-processing methods may suffer from the robustness degradation effect. A potential cause of this negative effect is that adversarial training examples are static and independent to the pre-processing model. We propose a method called Joint Adversarial Training based Pre-processing (JATP) defense.
  arXiv  Detail & Related papers  (2021-06-10T01:45:32Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Self-Progressing Robust Training [146.8337017922058]
  Current robust training methods such as adversarial training explicitly uses an "attack" to generate adversarial examples. We propose a new framework called SPROUT, self-progressing robust training. Our results shed new light on scalable, effective and attack-independent robust training methods.
  arXiv  Detail & Related papers  (2020-12-22T00:45:24Z)
• Boosting Adversarial Training with Hypersphere Embedding [53.75693100495097]
  Adversarial training is one of the most effective defenses against adversarial attacks for deep learning models. In this work, we advocate incorporating the hypersphere embedding mechanism into the AT procedure. We validate our methods under a wide range of adversarial attacks on the CIFAR-10 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-02-20T08:42:29Z)
• Improving the affordability of robustness training for DNNs [11.971637253035107]
  We show that the initial phase of adversarial training is redundant and can be replaced with natural training which significantly improves the computational efficiency. We show that our proposed method can reduce the training time by a factor of up to 2.5 with comparable or better model test accuracy and generalization on various strengths of adversarial attacks.
  arXiv  Detail & Related papers  (2020-02-11T07:29:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.