Exploring Memorization in Adversarial Training
- URL: http://arxiv.org/abs/2106.01606v1
- Date: Thu, 3 Jun 2021 05:39:57 GMT
- Title: Exploring Memorization in Adversarial Training
- Authors: Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang, Zhijie Deng, Hang Su, Jun
Zhu
- Abstract summary: We investigate the memorization effect in adversarial training (AT) for promoting a deeper understanding of capacity, convergence, generalization, and especially robust overfitting.
We propose a new mitigation algorithm motivated by detailed memorization analyses.
- Score: 58.38336773082818
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: It is well known that deep learning models have a propensity for fitting the
entire training set even with random labels, which requires memorization of
every training sample. In this paper, we investigate the memorization effect in
adversarial training (AT) for promoting a deeper understanding of capacity,
convergence, generalization, and especially robust overfitting of adversarially
trained classifiers. We first demonstrate that deep networks have sufficient
capacity to memorize adversarial examples of training data with completely
random labels, but not all AT algorithms can converge under the extreme
circumstance. Our study of AT with random labels motivates further analyses on
the convergence and generalization of AT. We find that some AT methods suffer
from a gradient instability issue, and the recently suggested complexity
measures cannot explain robust generalization by considering models trained on
random labels. Furthermore, we identify a significant drawback of memorization
in AT that it could result in robust overfitting. We then propose a new
mitigation algorithm motivated by detailed memorization analyses. Extensive
experiments on various datasets validate the effectiveness of the proposed
method.
Related papers
- Predicting and analyzing memorization within fine-tuned Large Language Models [0.0]
Large Language Models memorize a significant proportion of their training data, posing a serious threat when disclosed at inference time.
We propose a new approach based on sliced mutual information to detect memorized samples a priori.
We obtain strong empirical results, paving the way for systematic inspection and protection of these vulnerable samples before memorization happens.
arXiv Detail & Related papers (2024-09-27T15:53:55Z) - Unsupervised Continual Anomaly Detection with Contrastively-learned
Prompt [80.43623986759691]
We introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD.
The framework equips the UAD with continual learning capability through contrastively-learned prompts.
We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation.
arXiv Detail & Related papers (2024-01-02T03:37:11Z) - Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice.
We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
arXiv Detail & Related papers (2023-12-27T09:03:43Z) - Unintended Memorization in Large ASR Models, and How to Mitigate It [16.047859326721046]
auditing memorization in large non-auto-regressive automatic speech recognition (ASR) models has been challenging.
We design a simple auditing method to measure memorization in large ASR models without the extra compute overhead.
We show that in large-scale distributed training, clipping the average gradient on each compute core maintains neutral model quality and compute cost.
arXiv Detail & Related papers (2023-10-18T06:45:49Z) - Contrastive Deep Encoding Enables Uncertainty-aware
Machine-learning-assisted Histopathology [6.548275341067594]
terabytes of training data can be consciously utilized to pre-train deep networks to encode informative representations.
We show that our approach can reach the state-of-the-art (SOTA) for patch-level classification with only 1-10% randomly selected annotations.
arXiv Detail & Related papers (2023-09-13T17:37:19Z) - The Curious Case of Benign Memorization [19.74244993871716]
We show that under training protocols that include data augmentation, neural networks learn to memorize entirely random labels in a benign way.
We demonstrate that deep models have the surprising ability to separate noise from signal by distributing the task of memorization and feature learning to different layers.
arXiv Detail & Related papers (2022-10-25T13:41:31Z) - Learning and Certification under Instance-targeted Poisoning [49.55596073963654]
We study PAC learnability and certification under instance-targeted poisoning attacks.
We show that when the budget of the adversary scales sublinearly with the sample complexity, PAC learnability and certification are achievable.
We empirically study the robustness of K nearest neighbour, logistic regression, multi-layer perceptron, and convolutional neural network on real data sets.
arXiv Detail & Related papers (2021-05-18T17:48:15Z) - Deep Semi-supervised Knowledge Distillation for Overlapping Cervical
Cell Instance Segmentation [54.49894381464853]
We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation.
We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining.
Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
arXiv Detail & Related papers (2020-07-21T13:27:09Z) - Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions.
We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.
We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
arXiv Detail & Related papers (2020-06-13T08:24:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.