Frequency Regularization for Improving Adversarial Robustness
- URL: http://arxiv.org/abs/2212.12732v1
- Date: Sat, 24 Dec 2022 13:14:45 GMT
- Title: Frequency Regularization for Improving Adversarial Robustness
- Authors: Binxiao Huang, Chaofan Tao, Rui Lin, Ngai Wong
- Abstract summary: adversarial training (AT) has proven to be an effective defense approach.
We propose a frequency regularization (FR) to align the output difference in the spectral domain.
We find that our method achieves the strongest robustness against attacks by PGD-20, C&W and Autoattack.
- Score: 8.912245110734334
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep neural networks are incredibly vulnerable to crafted,
human-imperceptible adversarial perturbations. Although adversarial training
(AT) has proven to be an effective defense approach, we find that the
AT-trained models heavily rely on the input low-frequency content for judgment,
accounting for the low standard accuracy. To close the large gap between the
standard and robust accuracies during AT, we investigate the frequency
difference between clean and adversarial inputs, and propose a frequency
regularization (FR) to align the output difference in the spectral domain.
Besides, we find Stochastic Weight Averaging (SWA), by smoothing the kernels
over epochs, further improves the robustness. Among various defense schemes,
our method achieves the strongest robustness against attacks by PGD-20, C\&W
and Autoattack, on a WideResNet trained on CIFAR-10 without any extra data.
Related papers
- The Effectiveness of Random Forgetting for Robust Generalization [21.163070161951868]
We introduce a novel learning paradigm called "Forget to Mitigate Overfitting" (FOMO)
FOMO alternates between the forgetting phase, which randomly forgets a subset of weights, and the relearning phase, which emphasizes learning generalizable features.
Our experiments show that FOMO alleviates robust overfitting by significantly reducing the gap between the best and last robust test accuracy.
arXiv Detail & Related papers (2024-02-18T23:14:40Z) - Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
We propose a novel doubly-robust instance reweighted adversarial framework.
Our importance weights are obtained by optimizing the KL-divergence regularized loss function.
Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
arXiv Detail & Related papers (2023-08-01T06:16:18Z) - Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections [12.807619042576018]
We propose a novel test-time defense strategy called Robust Feature Inference (RFI)
RFI is easy to integrate with any existing (robust) training procedure without additional test-time computation.
We show that RFI improves robustness across adaptive and transfer attacks consistently.
arXiv Detail & Related papers (2023-07-21T16:18:58Z) - A Spectral Perspective towards Understanding and Improving Adversarial
Robustness [8.912245110734334]
adversarial training (AT) has proven to be an effective defense approach, but mechanism for robustness improvement is not fully understood.
We show that AT induces the deep model to focus more on the low-frequency region, which retains the shape-biased representations, to gain robustness.
We propose a spectral alignment regularization (SAR) such that the spectral output inferred by an attacked adversarial input stays as close as possible to its natural input counterpart.
arXiv Detail & Related papers (2023-06-25T14:47:03Z) - RUSH: Robust Contrastive Learning via Randomized Smoothing [31.717748554905015]
In this paper, we show a surprising fact that contrastive pre-training has an interesting yet implicit connection with robustness.
We design a powerful robust algorithm against adversarial attacks, RUSH, that combines the standard contrastive pre-training and randomized smoothing.
Our work has an improvement of over 15% in robust accuracy and a slight improvement in standard accuracy, compared to the state-of-the-arts.
arXiv Detail & Related papers (2022-07-11T18:45:14Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Interpolated Joint Space Adversarial Training for Robust and
Generalizable Defenses [82.3052187788609]
Adversarial training (AT) is considered to be one of the most reliable defenses against adversarial attacks.
Recent works show generalization improvement with adversarial samples under novel threat models.
We propose a novel threat model called Joint Space Threat Model (JSTM)
Under JSTM, we develop novel adversarial attacks and defenses.
arXiv Detail & Related papers (2021-12-12T21:08:14Z) - Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications.
We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths.
Our method is trained to automatically align features of arbitrary attacking strength.
arXiv Detail & Related papers (2021-05-31T17:01:05Z) - Combating Adversaries with Anti-Adversaries [118.70141983415445]
In particular, our layer generates an input perturbation in the opposite direction of the adversarial one.
We verify the effectiveness of our approach by combining our layer with both nominally and robustly trained models.
Our anti-adversary layer significantly enhances model robustness while coming at no cost on clean accuracy.
arXiv Detail & Related papers (2021-03-26T09:36:59Z) - Multiplicative Reweighting for Robust Neural Network Optimization [51.67267839555836]
Multiplicative weight (MW) updates are robust to moderate data corruptions in expert advice.
We show that MW improves the accuracy of neural networks in the presence of label noise.
arXiv Detail & Related papers (2021-02-24T10:40:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.