Self-paced Resistance Learning against Overfitting on Noisy Labels

• URL: http://arxiv.org/abs/2105.03059v1
• Date: Fri, 7 May 2021 04:17:20 GMT
• Title: Self-paced Resistance Learning against Overfitting on Noisy Labels
• Authors: Xiaoshuang Shi, Zhenhua Guo, Fuyong Xing, Yun Liang, Xiaofeng Zhu
• Abstract summary: Deep neural networks might first memorize the probably correct-label data and then corrupt-label samples. We propose a novel yet simple self-paced resistance framework to resist corrupted labels.
• Score: 25.916498598323667
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Noisy labels composed of correct and corrupted ones are pervasive in practice. They might significantly deteriorate the performance of convolutional neural networks (CNNs), because CNNs are easily overfitted on corrupted labels. To address this issue, inspired by an observation, deep neural networks might first memorize the probably correct-label data and then corrupt-label samples, we propose a novel yet simple self-paced resistance framework to resist corrupted labels, without using any clean validation data. The proposed framework first utilizes the memorization effect of CNNs to learn a curriculum, which contains confident samples and provides meaningful supervision for other training samples. Then it adopts selected confident samples and a proposed resistance loss to update model parameters; the resistance loss tends to smooth model parameters' update or attain equivalent prediction over each class, thereby resisting model overfitting on corrupted labels. Finally, we unify these two modules into a single loss function and optimize it in an alternative learning. Extensive experiments demonstrate the significantly superior performance of the proposed framework over recent state-of-the-art methods on noisy-label data. Source codes of the proposed method are available on https://github.com/xsshi2015/Self-paced-Resistance-Learning.

Related papers

• Verifix: Post-Training Correction to Improve Label Noise Robustness with Verified Samples [9.91998873101083]
  Post-Training Correction adjusts model parameters after initial training to mitigate label noise. We introduce Verifix, a novel algorithm that leverages a small, verified dataset to correct the model weights using a single update. Experiments on the CIFAR dataset with 25% synthetic corruption show 7.36% generalization improvements on average.
  arXiv  Detail & Related papers  (2024-03-13T15:32:08Z)
• Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
  Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference. One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels. This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise. We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
  arXiv  Detail & Related papers  (2024-01-15T03:33:39Z)
• Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets [11.385682758047775]
  One of the weaknesses of self-training is the semantic drift problem. We reshape the role of pseudo-labels and create a hierarchical order of information. A crucial step in self-training is to use the confidence prediction to select the best candidate pseudo-labels.
  arXiv  Detail & Related papers  (2023-12-31T19:25:34Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Combating Label Noise With A General Surrogate Model For Sample Selection [84.61367781175984]
  We propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically. We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets.
  arXiv  Detail & Related papers  (2023-10-16T14:43:27Z)
• Knockoffs-SPR: Clean Sample Selection in Learning with Noisy Labels [56.81761908354718]
  We propose a novel theoretically guaranteed clean sample selection framework for learning with noisy labels. Knockoffs-SPR can be regarded as a sample selection module for a standard supervised training pipeline. We further combine it with a semi-supervised algorithm to exploit the support of noisy data as unlabeled data.
  arXiv  Detail & Related papers  (2023-01-02T07:13:28Z)
• Context-based Virtual Adversarial Training for Text Classification with Noisy Labels [1.9508698179748525]
  We propose context-based virtual adversarial training (ConVAT) to prevent a text classifier from overfitting to noisy labels. Unlike the previous works, the proposed method performs the adversarial training at the context level rather than the inputs. We conduct extensive experiments on four text classification datasets with two types of label noises.
  arXiv  Detail & Related papers  (2022-05-29T14:19:49Z)
• LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification [28.37907856670151]
  Pseudo-labels are noisy due to their nature, so selecting the correct ones has a huge potential for performance boost. We propose a novel pseudo-label selection method LOPS that memorize takes learning order of samples into consideration. LOPS can be viewed as a strong performance-boost plug-in to most of existing weakly-supervised text classification methods.
  arXiv  Detail & Related papers  (2022-05-25T06:46:48Z)
• UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning [89.56465237941013]
  We propose UNICON, a simple yet effective sample selection method which is robust to high label noise. We obtain an 11.4% improvement over the current state-of-the-art on CIFAR100 dataset with a 90% noise rate.
  arXiv  Detail & Related papers  (2022-03-28T07:36:36Z)
• Synergistic Network Learning and Label Correction for Noise-robust Image Classification [28.27739181560233]
  Deep Neural Networks (DNNs) tend to overfit training label noise, resulting in poorer model performance in practice. We propose a robust label correction framework combining the ideas of small loss selection and noise correction. We demonstrate our method on both synthetic and real-world datasets with different noise types and rates.
  arXiv  Detail & Related papers  (2022-02-27T23:06:31Z)
• Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
  Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions. We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples. We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
  arXiv  Detail & Related papers  (2020-06-13T08:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.