Related papers: More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models

More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models

URL: http://arxiv.org/abs/2002.04725v3
Date: Sat, 15 Aug 2020 23:36:51 GMT
Title: More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
Authors: Lin Chen, Yifei Min, Mingrui Zhang, Amin Karbasi
Abstract summary: Modern machine learning models are susceptible to adversarial attacks that make human-imperceptibles to the data, but result in serious and potentially dangerous prediction errors. To address this issue, practitioners often use adversarial training to learn models that are robust against such attacks at the cost of higher generalization error on unperturbed test sets. We study the training of robust classifiers for both Gaussian and Bernoulli models under $ell_infty$ attacks, and we prove that more data may actually increase this gap.
Score: 37.85664398110855
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Despite remarkable success in practice, modern machine learning models have been found to be susceptible to adversarial attacks that make human-imperceptible perturbations to the data, but result in serious and potentially dangerous prediction errors. To address this issue, practitioners often use adversarial training to learn models that are robust against such attacks at the cost of higher generalization error on unperturbed test sets. The conventional wisdom is that more training data should shrink the gap between the generalization error of adversarially-trained models and standard models. However, we study the training of robust classifiers for both Gaussian and Bernoulli models under $\ell_\infty$ attacks, and we prove that more data may actually increase this gap. Furthermore, our theoretical results identify if and when additional data will finally begin to shrink the gap. Lastly, we experimentally demonstrate that our results also hold for linear regression models, which may indicate that this phenomenon occurs more broadly.

Related papers

Overparameterized Linear Regression under Adversarial Attacks [0.0]
We study the error of linear regression in the face of adversarial attacks. We show that adding features to linear models might be either a source of additional robustness or brittleness.
arXiv Detail & Related papers (2022-04-13T09:50:41Z)
Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference [29.743945643424553]
Over parameterized models generalize well (small error on the test data) even when trained to memorize the training data (zero error on the training data) This has led to an arms race towards increasingly over parameterized models (c.f., deep learning)
arXiv Detail & Related papers (2022-02-02T19:00:21Z)
Learning to Learn Transferable Attack [77.67399621530052]
Transfer adversarial attack is a non-trivial black-box adversarial attack that aims to craft adversarial perturbations on the surrogate model and then apply such perturbations to the victim model. We propose a Learning to Learn Transferable Attack (LLTA) method, which makes the adversarial perturbations more generalized via learning from both data and model augmentation. Empirical results on the widely-used dataset demonstrate the effectiveness of our attack method with a 12.85% higher success rate of transfer attack compared with the state-of-the-art methods.
arXiv Detail & Related papers (2021-12-10T07:24:21Z)
Universal Adversarial Attack on Deep Learning Based Prognostics [0.0]
We present the concept of universal adversarial perturbation, a special imperceptible noise to fool regression based RUL prediction models. We show that addition of universal adversarial perturbation to any instance of the input data increases error in the output predicted by the model. We further demonstrate the effect of varying the strength of perturbations on RUL prediction models and found that model accuracy decreases with the increase in perturbation strength.
arXiv Detail & Related papers (2021-09-15T08:05:16Z)
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study [65.17429512679695]
In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Despite ADC's intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models.
arXiv Detail & Related papers (2021-06-02T00:48:33Z)
MixKD: Towards Efficient Distillation of Large-scale Language Models [129.73786264834894]
We propose MixKD, a data-agnostic distillation framework, to endow the resulting model with stronger generalization ability. We prove from a theoretical perspective that under reasonable conditions MixKD gives rise to a smaller gap between the error and the empirical error. Experiments under a limited-data setting and ablation studies further demonstrate the advantages of the proposed approach.
arXiv Detail & Related papers (2020-11-01T18:47:51Z)
Asymptotic Behavior of Adversarial Training in Binary Classification [41.7567932118769]
Adversarial training is considered to be the state-of-the-art method for defense against adversarial attacks. Despite being successful in practice, several problems in understanding performance of adversarial training remain open. We derive precise theoretical predictions for the minimization of adversarial training in binary classification.
arXiv Detail & Related papers (2020-10-26T01:44:20Z)
Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
arXiv Detail & Related papers (2020-06-22T21:12:31Z)
On the Benefits of Invariance in Neural Networks [56.362579457990094]
We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
arXiv Detail & Related papers (2020-05-01T02:08:58Z)
The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization [36.87923859576768]
Adversarial training has shown its ability in producing models that are robust to perturbations on the input data, but usually at the expense of decrease in the standard accuracy. In this paper, we show that more training data can hurt the generalization of adversarially robust models in the classification problems.
arXiv Detail & Related papers (2020-02-25T18:25:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.