Unleashing the Potential of Regularization Strategies in Learning with Noisy Labels

• URL: http://arxiv.org/abs/2307.05025v1
• Date: Tue, 11 Jul 2023 05:58:20 GMT
• Title: Unleashing the Potential of Regularization Strategies in Learning with Noisy Labels
• Authors: Hui Kang, Sheng Liu, Huaxi Huang, Jun Yu, Bo Han, Dadong Wang, Tongliang Liu
• Abstract summary: We demonstrate that a simple baseline using cross-entropy loss, combined with widely used regularization strategies can outperform state-of-the-art methods. Our findings suggest that employing a combination of regularization strategies can be more effective than intricate algorithms in tackling the challenges of learning with noisy labels.
• Score: 65.92994348757743
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, research on learning with noisy labels has focused on devising novel algorithms that can achieve robustness to noisy training labels while generalizing to clean data. These algorithms often incorporate sophisticated techniques, such as noise modeling, label correction, and co-training. In this study, we demonstrate that a simple baseline using cross-entropy loss, combined with widely used regularization strategies like learning rate decay, model weights average, and data augmentations, can outperform state-of-the-art methods. Our findings suggest that employing a combination of regularization strategies can be more effective than intricate algorithms in tackling the challenges of learning with noisy labels. While some of these regularization strategies have been utilized in previous noisy label learning research, their full potential has not been thoroughly explored. Our results encourage a reevaluation of benchmarks for learning with noisy labels and prompt reconsideration of the role of specialized learning algorithms designed for training with noisy labels.

Related papers

• DIRECT: Deep Active Learning under Imbalance and Label Noise [15.571923343398657]
  We conduct the first study of active learning under both class imbalance and label noise. We propose a novel algorithm that robustly identifies the class separation threshold and annotates the most uncertain examples. Our results demonstrate that DIRECT can save more than 60% of the annotation budget compared to state-of-art active learning algorithms.
  arXiv  Detail & Related papers  (2023-12-14T18:18:34Z)
• ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerance [53.73316938815873]
  We propose a method called ERASE (Error-Resilient representation learning on graphs for lAbel noiSe tolerancE) to learn representations with error tolerance. ERASE combines prototype pseudo-labels with propagated denoised labels and updates representations with error resilience. Our method can outperform multiple baselines with clear margins in broad noise levels and enjoy great scalability.
  arXiv  Detail & Related papers  (2023-12-13T17:59:07Z)
• Learning with Neighbor Consistency for Noisy Labels [69.83857578836769]
  We present a method for learning from noisy labels that leverages similarities between training examples in feature space. We evaluate our method on datasets evaluating both synthetic (CIFAR-10, CIFAR-100) and realistic (mini-WebVision, Clothing1M, mini-ImageNet-Red) noise.
  arXiv  Detail & Related papers  (2022-02-04T15:46:27Z)
• Robust Long-Tailed Learning under Label Noise [50.00837134041317]
  This work investigates the label noise problem under long-tailed label distribution. We propose a robust framework,algo, that realizes noise detection for long-tailed learning. Our framework can naturally leverage semi-supervised learning algorithms to further improve the generalisation.
  arXiv  Detail & Related papers  (2021-08-26T03:45:00Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)
• A Survey on Deep Learning with Noisy Labels: How to train your model when you cannot trust on the annotations? [21.562089974755125]
  Several approaches have been proposed to improve the training of deep learning models in the presence of noisy labels. This paper presents a survey on the main techniques in literature, in which we classify the algorithm in the following groups: robust losses, sample weighting, sample selection, meta-learning, and combined approaches.
  arXiv  Detail & Related papers  (2020-12-05T15:45:20Z)
• Semi-Supervised Learning with Meta-Gradient [123.26748223837802]
  We propose a simple yet effective meta-learning algorithm in semi-supervised learning. We find that the proposed algorithm performs favorably against state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-08T08:48:56Z)
• Early-Learning Regularization Prevents Memorization of Noisy Labels [29.04549895470588]
  We propose a novel framework to perform classification via deep learning in the presence of noisy annotations. Deep neural networks have been observed to first fit the training data with clean labels during an "early learning" phase. We design a regularization term that steers the model towards these targets, implicitly preventing memorization of the false labels.
  arXiv  Detail & Related papers  (2020-06-30T23:46:33Z)
• Noise-tolerant, Reliable Active Classification with Comparison Queries [25.725730509014355]
  We study the paradigm of active learning, in which algorithms with access to large pools of data may adaptively choose what samples to label. We provide the first time and query efficient algorithms for learning non-homogeneous linear separators robust to bounded (Massart) noise.
  arXiv  Detail & Related papers  (2020-01-15T19:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.