How benign is benign overfitting?
- URL: http://arxiv.org/abs/2007.04028v1
- Date: Wed, 8 Jul 2020 11:07:10 GMT
- Title: How benign is benign overfitting?
- Authors: Amartya Sanyal, Puneet K Dokania, Varun Kanade, Philip H.S. Torr
- Abstract summary: We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models.
Deep neural networks essentially achieve zero training error, even in the presence of label noise.
We identify label noise as one of the causes for adversarial vulnerability.
- Score: 96.07549886487526
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We investigate two causes for adversarial vulnerability in deep neural
networks: bad data and (poorly) trained models. When trained with SGD, deep
neural networks essentially achieve zero training error, even in the presence
of label noise, while also exhibiting good generalization on natural test data,
something referred to as benign overfitting [2, 10]. However, these models are
vulnerable to adversarial attacks. We identify label noise as one of the causes
for adversarial vulnerability, and provide theoretical and empirical evidence
in support of this. Surprisingly, we find several instances of label noise in
datasets such as MNIST and CIFAR, and that robustly trained models incur
training error on some of these, i.e. they don't fit the noise. However,
removing noisy labels alone does not suffice to achieve adversarial robustness.
Standard training procedures bias neural networks towards learning "simple"
classification boundaries, which may be less robust than more complex ones. We
observe that adversarial training does produce more complex decision
boundaries. We conjecture that in part the need for complex decision boundaries
arises from sub-optimal representation learning. By means of simple toy
examples, we show theoretically how the choice of representation can
drastically affect adversarial robustness.
Related papers
- Theoretical Understanding of Learning from Adversarial Perturbations [30.759348459463467]
It is not fully understood why adversarial examples can deceive neural networks and transfer between different networks.
We provide a theoretical framework for understanding learning from perturbations using a one-hidden-layer network.
Our results highlight that various adversarial perturbations, even perturbations of a few pixels, contain sufficient class features for generalization.
arXiv Detail & Related papers (2024-02-16T06:22:44Z) - F$^2$AT: Feature-Focusing Adversarial Training via Disentanglement of
Natural and Perturbed Patterns [74.03108122774098]
Deep neural networks (DNNs) are vulnerable to adversarial examples crafted by well-designed perturbations.
This could lead to disastrous results on critical applications such as self-driving cars, surveillance security, and medical diagnosis.
We propose a Feature-Focusing Adversarial Training (F$2$AT) which enforces the model to focus on the core features from natural patterns.
arXiv Detail & Related papers (2023-10-23T04:31:42Z) - Benign Overfitting for Two-layer ReLU Convolutional Neural Networks [60.19739010031304]
We establish algorithm-dependent risk bounds for learning two-layer ReLU convolutional neural networks with label-flipping noise.
We show that, under mild conditions, the neural network trained by gradient descent can achieve near-zero training loss and Bayes optimal test risk.
arXiv Detail & Related papers (2023-03-07T18:59:38Z) - Adversarial Noises Are Linearly Separable for (Nearly) Random Neural
Networks [46.13404040937189]
Adversarial examples, which are usually generated for specific inputs with a specific model, are ubiquitous for neural networks.
In this paper we unveil a surprising property of adversarial noises when they are put together, i.e., adversarial noises crafted by one-step methods are linearly separable if equipped with the corresponding labels.
arXiv Detail & Related papers (2022-06-09T07:26:46Z) - Robust Training under Label Noise by Over-parameterization [41.03008228953627]
We propose a principled approach for robust training of over-parameterized deep networks in classification tasks where a proportion of training labels are corrupted.
The main idea is yet very simple: label noise is sparse and incoherent with the network learned from clean data, so we model the noise and learn to separate it from the data.
Remarkably, when trained using such a simple method in practice, we demonstrate state-of-the-art test accuracy against label noise on a variety of real datasets.
arXiv Detail & Related papers (2022-02-28T18:50:10Z) - Benign Overfitting in Adversarially Robust Linear Classification [91.42259226639837]
"Benign overfitting", where classifiers memorize noisy training data yet still achieve a good generalization performance, has drawn great attention in the machine learning community.
We show that benign overfitting indeed occurs in adversarial training, a principled approach to defend against adversarial examples.
arXiv Detail & Related papers (2021-12-31T00:27:31Z) - A Good Representation Detects Noisy Labels [9.4092903583089]
Label noise is pervasive in real-world datasets, which encodes wrong correlation patterns and impairs the generalization of deep neural networks (DNNs)
We propose a universally applicable and trainingfree solution to detect noisy labels.
Experiments with both synthetic and real-world label noise demonstrate our training-free solutions are significantly improving over most of the training-based datasets.
arXiv Detail & Related papers (2021-10-12T19:10:30Z) - Property-driven Training: All You (N)Ever Wanted to Know About [0.0]
Neural networks are known for their ability to detect general patterns in noisy data.
This makes them a popular tool for perception components in complex AI systems.
adversarial training, data-augmentation and Lipschitz robustness training have been proposed as means of improving their robustness.
arXiv Detail & Related papers (2021-04-03T13:06:06Z) - Tackling Instance-Dependent Label Noise via a Universal Probabilistic
Model [80.91927573604438]
This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances.
Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
arXiv Detail & Related papers (2021-01-14T05:43:51Z) - Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions.
We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.
We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
arXiv Detail & Related papers (2020-06-13T08:24:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.