Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization

• URL: http://arxiv.org/abs/2401.16352v4
• Date: Fri, 23 Aug 2024 06:08:14 GMT
• Title: Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization
• Authors: Guang Lin, Chao Li, Jianhai Zhang, Toshihisa Tanaka, Qibin Zhao,
• Abstract summary: We propose a novel pipeline to acquire the robust purifier model, named Adversarial Training on Purification (AToP) To evaluate our method in an efficient and scalable way, we conduct extensive experiments on CIFAR-10, CIFAR-100, and ImageNette.
• Score: 29.09894840783714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The deep neural networks are known to be vulnerable to well-designed adversarial attacks. The most successful defense technique based on adversarial training (AT) can achieve optimal robustness against particular attacks but cannot generalize well to unseen attacks. Another effective defense technique based on adversarial purification (AP) can enhance generalization but cannot achieve optimal robustness. Meanwhile, both methods share one common limitation on the degraded standard accuracy. To mitigate these issues, we propose a novel pipeline to acquire the robust purifier model, named Adversarial Training on Purification (AToP), which comprises two components: perturbation destruction by random transforms (RT) and purifier model fine-tuned (FT) by adversarial loss. RT is essential to avoid overlearning to known attacks, resulting in the robustness generalization to unseen attacks, and FT is essential for the improvement of robustness. To evaluate our method in an efficient and scalable way, we conduct extensive experiments on CIFAR-10, CIFAR-100, and ImageNette to demonstrate that our method achieves optimal robustness and exhibits generalization ability against unseen attacks.

Related papers

• How Robust Are Energy-Based Models Trained With Equilibrium Propagation? [4.374837991804085]
  Adrial training is the current state-of-the-art defense against adversarial attacks. It lowers the model's accuracy on clean inputs, is computationally expensive, and offers less robustness to natural noise. In contrast, energy-based models (EBMs) incorporate feedback connections from each layer to the previous layer, yielding a recurrent, deep-attractor architecture.
  arXiv  Detail & Related papers  (2024-01-21T16:55:40Z)
• Guidance Through Surrogate: Towards a Generic Diagnostic Attack [101.36906370355435]
  We develop a guided mechanism to avoid local minima during attack optimization, leading to a novel attack dubbed Guided Projected Gradient Attack (G-PGA) Our modified attack does not require random restarts, large number of attack iterations or search for an optimal step-size. More than an effective attack, G-PGA can be used as a diagnostic tool to reveal elusive robustness due to gradient masking in adversarial defenses.
  arXiv  Detail & Related papers  (2022-12-30T18:45:23Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients [65.52888259961803]
  We introduce ROPUST, a simple and efficient method to leverage robust pre-trained models and increase their robustness. We test our method on nine different models against four attacks in RobustBench, consistently improving over state-of-the-art performance. We show that even with state-of-the-art phase retrieval techniques, ROPUST remains an effective defense.
  arXiv  Detail & Related papers  (2021-07-06T12:03:36Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses [59.58128343334556]
  We introduce a relaxation term to the standard loss, that finds more suitable gradient-directions, increases attack efficacy and leads to more efficient adversarial training. We propose Guided Adversarial Margin Attack (GAMA), which utilizes function mapping of the clean image to guide the generation of adversaries. We also propose Guided Adversarial Training (GAT), which achieves state-of-the-art performance amongst single-step defenses.
  arXiv  Detail & Related papers  (2020-11-30T16:39:39Z)
• Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models [82.03536496686763]
  The vulnerability of deep networks to adversarial attacks is a central problem for deep learning from the perspective of both cognition and security. We focus on defending naturally-trained classifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Based Model (EBM) for adversarial purification. Our contributions are 1) an improved method for training EBM's with realistic long-run MCMC samples, 2) Expectation-Over-Transformation (EOT) defense that resolves theoretical ambiguities for defenses, and 3) state-of-the-art adversarial defense for naturally-trained classifiers and competitive defense.
  arXiv  Detail & Related papers  (2020-05-27T17:53:36Z)
• Improving the affordability of robustness training for DNNs [11.971637253035107]
  We show that the initial phase of adversarial training is redundant and can be replaced with natural training which significantly improves the computational efficiency. We show that our proposed method can reduce the training time by a factor of up to 2.5 with comparable or better model test accuracy and generalization on various strengths of adversarial attacks.
  arXiv  Detail & Related papers  (2020-02-11T07:29:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.