The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization

• URL: http://arxiv.org/abs/2002.11080v2
• Date: Fri, 5 Jun 2020 23:46:22 GMT
• Title: The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• Authors: Yifei Min, Lin Chen, Amin Karbasi
• Abstract summary: Adversarial training has shown its ability in producing models that are robust to perturbations on the input data, but usually at the expense of decrease in the standard accuracy. In this paper, we show that more training data can hurt the generalization of adversarially robust models in the classification problems.
• Score: 36.87923859576768
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial training has shown its ability in producing models that are robust to perturbations on the input data, but usually at the expense of decrease in the standard accuracy. To mitigate this issue, it is commonly believed that more training data will eventually help such adversarially robust models generalize better on the benign/unperturbed test data. In this paper, however, we challenge this conventional belief and show that more training data can hurt the generalization of adversarially robust models in the classification problems. We first investigate the Gaussian mixture classification with a linear loss and identify three regimes based on the strength of the adversary. In the weak adversary regime, more data improves the generalization of adversarially robust models. In the medium adversary regime, with more training data, the generalization loss exhibits a double descent curve, which implies the existence of an intermediate stage where more training data hurts the generalization. In the strong adversary regime, more data almost immediately causes the generalization error to increase. Then we move to the analysis of a two-dimensional classification problem with a 0-1 loss. We prove that more data always hurts the generalization performance of adversarially trained models with large perturbations. To complement our theoretical results, we conduct empirical studies on Gaussian mixture classification, support vector machines (SVMs), and linear regression.

Related papers

• Data-Dependent Stability Analysis of Adversarial Training [18.686469222136854]
  Adrial training is the most widely used defense against adversarial example attacks. Previous training for training have not included data distribution information. Our findings demonstrate distribution shifts from data poisoning generalization.
  arXiv  Detail & Related papers  (2024-01-06T08:18:04Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Why adversarial training can hurt robust accuracy [7.906608953906889]
  adversarial training helps when enough data is available, it may hurt robust generalization in the small sample size regime. Our proof provides explanatory insights that may also transfer to feature learning models.
  arXiv  Detail & Related papers  (2022-03-03T20:41:38Z)
• Guided Interpolation for Adversarial Training [73.91493448651306]
  As training progresses, the training data becomes less and less attackable, undermining the robustness enhancement. We propose the guided framework (GIF), which employs the previous epoch's meta information to guide the data's adversarial variants. Compared with the vanilla mixup, the GIF can provide a higher ratio of attackable data, which is beneficial to the robustness enhancement.
  arXiv  Detail & Related papers  (2021-02-15T03:55:08Z)
• Asymptotic Behavior of Adversarial Training in Binary Classification [41.7567932118769]
  Adversarial training is considered to be the state-of-the-art method for defense against adversarial attacks. Despite being successful in practice, several problems in understanding performance of adversarial training remain open. We derive precise theoretical predictions for the minimization of adversarial training in binary classification.
  arXiv  Detail & Related papers  (2020-10-26T01:44:20Z)
• Precise Statistical Analysis of Classification Accuracies for Adversarial Training [43.25761725062367]
  A variety of recent adversarial training procedures have been proposed to remedy this issue. We derive a precise characterization of the standard and robust accuracy for a class of minimax adversarially trained models.
  arXiv  Detail & Related papers  (2020-10-21T18:00:53Z)
• Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization [50.53690793828442]
  We show that both data whitening and second order optimization can harm or entirely prevent generalization. For a general class of models, namely models with a fully connected first layer, we prove that the information contained in this matrix is the only information which can be used to generalize.
  arXiv  Detail & Related papers  (2020-08-17T18:00:05Z)
• On the Benefits of Invariance in Neural Networks [56.362579457990094]
  We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
  arXiv  Detail & Related papers  (2020-05-01T02:08:58Z)
• Precise Tradeoffs in Adversarial Training for Linear Regression [55.764306209771405]
  We provide a precise and comprehensive understanding of the role of adversarial training in the context of linear regression with Gaussian features. We precisely characterize the standard/robust accuracy and the corresponding tradeoff achieved by a contemporary mini-max adversarial training approach. Our theory for adversarial training algorithms also facilitates the rigorous study of how a variety of factors (size and quality of training data, model overparametrization etc.) affect the tradeoff between these two competing accuracies.
  arXiv  Detail & Related papers  (2020-02-24T19:01:47Z)
• More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models [37.85664398110855]
  Modern machine learning models are susceptible to adversarial attacks that make human-imperceptibles to the data, but result in serious and potentially dangerous prediction errors. To address this issue, practitioners often use adversarial training to learn models that are robust against such attacks at the cost of higher generalization error on unperturbed test sets. We study the training of robust classifiers for both Gaussian and Bernoulli models under $ell_infty$ attacks, and we prove that more data may actually increase this gap.
  arXiv  Detail & Related papers  (2020-02-11T23:01:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.