Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training

• URL: http://arxiv.org/abs/2010.08034v1
• Date: Thu, 15 Oct 2020 21:43:07 GMT
• Title: Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training
• Authors: Zichao Li and Liyuan Liu and Chengyu Dong and Jingbo Shang
• Abstract summary: We propose APART, an adaptive adversarial training framework, which parameterizes perturbation generation and progressively strengthens them. APART provides comparable or even better robustness than PGD-10, with only about 1/4 of its computational cost.
• Score: 34.83228408320053
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our goal is to understand why the robustness drops after conducting adversarial training for too long. Although this phenomenon is commonly explained as overfitting, our analysis suggest that its primary cause is perturbation underfitting. We observe that after training for too long, FGSM-generated perturbations deteriorate into random noise. Intuitively, since no parameter updates are made to strengthen the perturbation generator, once this process collapses, it could be trapped in such local optima. Also, sophisticating this process could mostly avoid the robustness drop, which supports that this phenomenon is caused by underfitting instead of overfitting. In the light of our analyses, we propose APART, an adaptive adversarial training framework, which parameterizes perturbation generation and progressively strengthens them. Shielding perturbations from underfitting unleashes the potential of our framework. In our experiments, APART provides comparable or even better robustness than PGD-10, with only about 1/4 of its computational cost.

Related papers

• Adversarial Robustness Overestimation and Instability in TRADES [4.063518154926961]
  TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking.
  arXiv  Detail & Related papers  (2024-10-10T07:32:40Z)
• Extreme Miscalibration and the Illusion of Adversarial Robustness [66.29268991629085]
  Adversarial Training is often used to increase model robustness. We show that this observed gain in robustness is an illusion of robustness (IOR) We urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations.
  arXiv  Detail & Related papers  (2024-02-27T13:49:12Z)
• Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? [58.942118128503104]
  Causal confusion is a phenomenon where an agent learns a policy that reflects imperfect spurious correlations in the data. This phenomenon is particularly pronounced in domains such as robotics. In this paper, we study causal confusion in offline reinforcement learning.
  arXiv  Detail & Related papers  (2023-12-28T17:54:56Z)
• Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective [80.51463286812314]
  Adversarial Training (AT) has become arguably the state-of-the-art algorithm for extracting robust features. AT suffers from severe robust overfitting problems, particularly after learning rate (LR) decay. We show how LR decay breaks the balance between the minimax game by empowering the trainer with a stronger memorization ability.
  arXiv  Detail & Related papers  (2023-10-30T09:00:11Z)
• Understanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularization [5.1024659285813785]
  Adrial training is prone to overfitting, and the cause is far from clear. We find that robust overfitting results from standard training, specifically the minimization of the clean loss. We propose a new regularizer to smooth the loss landscape by penalizing the weighted logits variation along the adversarial direction.
  arXiv  Detail & Related papers  (2022-12-09T16:55:30Z)
• Understanding Catastrophic Overfitting in Single-step Adversarial Training [9.560980936110234]
  "catastrophic overfitting" is a phenomenon in which the robust accuracy against projected gradient descent suddenly decreases to 0% after a few epochs. We propose a simple method that not only prevents catastrophic overfitting, but also overrides the belief that it is difficult to prevent multi-step adversarial attacks with single-step adversarial training.
  arXiv  Detail & Related papers  (2020-10-05T06:13:35Z)
• Overfitting in adversarially robust deep learning [86.11788847990783]
  We show that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting.
  arXiv  Detail & Related papers  (2020-02-26T15:40:50Z)
• Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality [74.0084803220897]
  Adversarial training is a popular method to give neural nets robustness against adversarial perturbations. We show convergence to low robust training loss for emphpolynomial width instead of exponential, under natural assumptions and with the ReLU activation.
  arXiv  Detail & Related papers  (2020-02-16T20:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.