A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise

• URL: http://arxiv.org/abs/2003.06778v3
• Date: Thu, 12 Nov 2020 20:08:51 GMT
• Title: A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
• Authors: Mark Collier, Basil Mustafa, Efi Kokiopoulou, Rodolphe Jenatton, Jesse Berent
• Abstract summary: We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label noise. For image segmentation, our method increases the mean IoU on the PASCAL VOC and Cityscapes datasets by more than 1% over the state-of-the-art model.
• Score: 13.800625301418341
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Datasets with noisy labels are a common occurrence in practical applications of classification methods. We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label noise. We assume an underlying heteroscedastic generative process for noisy labels. To make gradient based training feasible we use a temperature parameterized softmax as a smooth approximation to the assumed generative process. We illustrate that the softmax temperature controls a bias-variance trade-off for the approximation. By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise. For image segmentation, our method increases the mean IoU on the PASCAL VOC and Cityscapes datasets by more than 1% over the state-of-the-art model.

Related papers

• Efficient Adaptive Label Refinement for Label Noise Learning [14.617885790129336]
  We propose Adaptive Label Refinement (ALR) to avoid incorrect labels and thoroughly learning clean samples. ALR is simple and efficient, requiring no prior knowledge of noise or auxiliary datasets. We validate ALR's effectiveness through experiments on benchmark datasets with artificial label noise (CIFAR-10/100) and real-world datasets with inherent noise (ANIMAL-10N, Clothing1M, WebVision)
  arXiv  Detail & Related papers  (2025-02-01T09:58:08Z)
• LaplaceConfidence: a Graph-based Approach for Learning with Noisy Labels [17.66525177980075]
  We introduce LaplaceConfidence, a method to obtain label confidence (i.e., clean probabilities) utilizing the Laplacian energy. LaplaceConfidence is embedded into a holistic method for robust training, where co-training technique generates unbiased label confidence. Our experiments demonstrate that LaplaceConfidence outperforms state-of-the-art methods on benchmark datasets under both synthetic and real-world noise.
  arXiv  Detail & Related papers  (2023-07-31T12:44:30Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Latent Class-Conditional Noise Model [54.56899309997246]
  We introduce a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples.
  arXiv  Detail & Related papers  (2023-02-19T15:24:37Z)
• Neighborhood Collective Estimation for Noisy Label Identification and Correction [92.20697827784426]
  Learning with noisy labels (LNL) aims at designing strategies to improve model performance and generalization by mitigating the effects of model overfitting to noisy labels. Recent advances employ the predicted label distributions of individual samples to perform noise verification and noisy label correction, easily giving rise to confirmation bias. We propose Neighborhood Collective Estimation, in which the predictive reliability of a candidate sample is re-estimated by contrasting it against its feature-space nearest neighbors.
  arXiv  Detail & Related papers  (2022-08-05T14:47:22Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels [86.5943044285146]
  The label noise transition matrix $T$ reflects the probabilities that true labels flip into noisy ones. In this paper, we focus on learning under the mixed closed-set and open-set label noise. Our method can better model the mixed label noise, following its more robust performance than the prior state-of-the-art label-noise learning methods.
  arXiv  Detail & Related papers  (2020-12-02T02:42:45Z)
• Error-Bounded Correction of Noisy Labels [17.510654621245656]
  We show that the prediction of a noisy classifier can indeed be a good indicator of whether the label of a training data is clean. Based on the theoretical result, we propose a novel algorithm that corrects the labels based on the noisy classifier prediction. We incorporate our label correction algorithm into the training of deep neural networks and train models that achieve superior testing performance on multiple public datasets.
  arXiv  Detail & Related papers  (2020-11-19T19:23:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.