Confidence Adaptive Regularization for Deep Learning with Noisy Labels

• URL: http://arxiv.org/abs/2108.08212v1
• Date: Wed, 18 Aug 2021 15:51:25 GMT
• Title: Confidence Adaptive Regularization for Deep Learning with Noisy Labels
• Authors: Yangdi Lu, Yang Bo, Wenbo He
• Abstract summary: Recent studies on the memorization effects of deep neural networks on noisy labels show that the networks first fit the correctly-labeled training samples before memorizing the mislabeled samples. Motivated by this early-learning phenomenon, we propose a novel method to prevent memorization of the mislabeled samples. We provide the theoretical analysis and conduct the experiments on synthetic and real-world datasets, demonstrating that our approach achieves comparable results to the state-of-the-art methods.
• Score: 2.0349696181833337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies on the memorization effects of deep neural networks on noisy labels show that the networks first fit the correctly-labeled training samples before memorizing the mislabeled samples. Motivated by this early-learning phenomenon, we propose a novel method to prevent memorization of the mislabeled samples. Unlike the existing approaches which use the model output to identify or ignore the mislabeled samples, we introduce an indicator branch to the original model and enable the model to produce a confidence value for each sample. The confidence values are incorporated in our loss function which is learned to assign large confidence values to correctly-labeled samples and small confidence values to mislabeled samples. We also propose an auxiliary regularization term to further improve the robustness of the model. To improve the performance, we gradually correct the noisy labels with a well-designed target estimation strategy. We provide the theoretical analysis and conduct the experiments on synthetic and real-world datasets, demonstrating that our approach achieves comparable results to the state-of-the-art methods.

Related papers

• Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection [82.43311784594384]
  Real-world datasets contain not only noisy labels but also class imbalance. We propose a simple yet effective method to address noisy labels in imbalanced datasets.
  arXiv  Detail & Related papers  (2024-02-17T10:34:53Z)
• Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
  Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference. One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels. This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise. We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
  arXiv  Detail & Related papers  (2024-01-15T03:33:39Z)
• Late Stopping: Avoiding Confidently Learning from Mislabeled Examples [61.00103151680946]
  We propose a new framework, Late Stopping, which leverages the intrinsic robust learning ability of DNNs through a prolonged training process. We empirically observe that mislabeled and clean examples exhibit differences in the number of epochs required for them to be consistently and correctly classified. Experimental results on benchmark-simulated and real-world noisy datasets demonstrate that the proposed method outperforms state-of-the-art counterparts.
  arXiv  Detail & Related papers  (2023-08-26T12:43:25Z)
• Learning with Noisy labels via Self-supervised Adversarial Noisy Masking [33.87292143223425]
  We propose a novel training approach termed adversarial noisy masking. It adaptively modulates the input data and label simultaneously, preventing the model to overfit noisy samples. It is tested on both synthetic and real-world noisy datasets.
  arXiv  Detail & Related papers  (2023-02-14T03:13:26Z)
• SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning [101.86916775218403]
  This paper revisits the popular pseudo-labeling methods via a unified sample weighting formulation. We propose SoftMatch to overcome the trade-off by maintaining both high quantity and high quality of pseudo-labels during training. In experiments, SoftMatch shows substantial improvements across a wide variety of benchmarks, including image, text, and imbalanced classification.
  arXiv  Detail & Related papers  (2023-01-26T03:53:25Z)
• Rethinking Precision of Pseudo Label: Test-Time Adaptation via Complementary Learning [10.396596055773012]
  We propose a novel complementary learning approach to enhance test-time adaptation. In test-time adaptation tasks, information from the source domain is typically unavailable. We highlight that the risk function of complementary labels agrees with their Vanilla loss formula.
  arXiv  Detail & Related papers  (2023-01-15T03:36:33Z)
• Label Noise-Robust Learning using a Confidence-Based Sieving Strategy [15.997774467236352]
  In learning tasks with label noise, improving model robustness against overfitting is a pivotal challenge. Identifying the samples with noisy labels and preventing the model from learning them is a promising approach to address this challenge. We propose a novel discriminator metric called confidence error and a sieving strategy called CONFES to differentiate between the clean and noisy samples effectively.
  arXiv  Detail & Related papers  (2022-10-11T10:47:28Z)
• Contrastive Regularization for Semi-Supervised Learning [46.020125061295886]
  We propose contrastive regularization to improve both efficiency and accuracy of the consistency regularization by well-clustered features of unlabeled data. Our method also shows robust performance on open-set semi-supervised learning where unlabeled data includes out-of-distribution samples.
  arXiv  Detail & Related papers  (2022-01-17T07:20:11Z)
• Semi-supervised Salient Object Detection with Effective Confidence Estimation [35.0990691497574]
  We study semi-supervised salient object detection with access to a small number of labeled samples and a large number of unlabeled samples. We model the nature of human saliency labels using the latent variable of the Conditional Energy-based Model. With only 1/16 labeled samples, our model achieves competitive performance compared with state-of-the-art fully-supervised models.
  arXiv  Detail & Related papers  (2021-12-28T07:14:48Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)
• Salvage Reusable Samples from Noisy Data for Robust Learning [70.48919625304]
  We propose a reusable sample selection and correction approach, termed as CRSSC, for coping with label noise in training deep FG models with web images. Our key idea is to additionally identify and correct reusable samples, and then leverage them together with clean examples to update the networks.
  arXiv  Detail & Related papers  (2020-08-06T02:07:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.