Harmless label noise and informative soft-labels in supervised classification

• URL: http://arxiv.org/abs/2104.02872v1
• Date: Wed, 7 Apr 2021 02:56:11 GMT
• Title: Harmless label noise and informative soft-labels in supervised classification
• Authors: Daniel Ahfock and Geoffrey J. McLachlan
• Abstract summary: Manual labelling of training examples is common practice in supervised learning. When the labelling task is of non-trivial difficulty, the supplied labels may not be equal to the ground-truth labels, and label noise is introduced into the training dataset. In particular, when classification difficulty is the only source of label errors, multiple sets of noisy labels can supply more information for the estimation of a classification rule.
• Score: 1.6752182911522517
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Manual labelling of training examples is common practice in supervised learning. When the labelling task is of non-trivial difficulty, the supplied labels may not be equal to the ground-truth labels, and label noise is introduced into the training dataset. If the manual annotation is carried out by multiple experts, the same training example can be given different class assignments by different experts, which is indicative of label noise. In the framework of model-based classification, a simple, but key observation is that when the manual labels are sampled using the posterior probabilities of class membership, the noisy labels are as valuable as the ground-truth labels in terms of statistical information. A relaxation of this process is a random effects model for imperfect labelling by a group that uses approximate posterior probabilities of class membership. The relative efficiency of logistic regression using the noisy labels compared to logistic regression using the ground-truth labels can then be derived. The main finding is that logistic regression can be robust to label noise when label noise and classification difficulty are positively correlated. In particular, when classification difficulty is the only source of label errors, multiple sets of noisy labels can supply more information for the estimation of a classification rule compared to the single set of ground-truth labels.

Related papers

• Learning with Confidence: Training Better Classifiers from Soft Labels [0.0]
  In supervised machine learning, models are typically trained using data with hard labels, i.e., definite assignments of class membership. We investigate whether incorporating label uncertainty, represented as discrete probability distributions over the class labels, improves the predictive performance of classification models.
  arXiv  Detail & Related papers  (2024-09-24T13:12:29Z)
• Extracting Clean and Balanced Subset for Noisy Long-tailed Classification [66.47809135771698]
  We develop a novel pseudo labeling method using class prototypes from the perspective of distribution matching. By setting a manually-specific probability measure, we can reduce the side-effects of noisy and long-tailed data simultaneously. Our method can extract this class-balanced subset with clean labels, which brings effective performance gains for long-tailed classification with label noise.
  arXiv  Detail & Related papers  (2024-04-10T07:34:37Z)
• Multi-Label Noise Transition Matrix Estimation with Label Correlations: Theory and Algorithm [73.94839250910977]
  Noisy multi-label learning has garnered increasing attention due to the challenges posed by collecting large-scale accurate labels. The introduction of transition matrices can help model multi-label noise and enable the development of statistically consistent algorithms. We propose a novel estimator that leverages label correlations without the need for anchor points or precise fitting of noisy class posteriors.
  arXiv  Detail & Related papers  (2023-09-22T08:35:38Z)
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces [64.23172847182109]
  We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. We provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance.
  arXiv  Detail & Related papers  (2021-05-12T15:40:13Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)
• Label Confusion Learning to Enhance Text Classification Models [3.0251266104313643]
  Label Confusion Model (LCM) learns label confusion to capture semantic overlap among labels. LCM can generate a better label distribution to replace the original one-hot label vector. experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models.
  arXiv  Detail & Related papers  (2020-12-09T11:34:35Z)
• Error-Bounded Correction of Noisy Labels [17.510654621245656]
  We show that the prediction of a noisy classifier can indeed be a good indicator of whether the label of a training data is clean. Based on the theoretical result, we propose a novel algorithm that corrects the labels based on the noisy classifier prediction. We incorporate our label correction algorithm into the training of deep neural networks and train models that achieve superior testing performance on multiple public datasets.
  arXiv  Detail & Related papers  (2020-11-19T19:23:23Z)
• Exploiting Context for Robustness to Label Noise in Active Learning [47.341705184013804]
  We address the problems of how a system can identify which of the queried labels are wrong and how a multi-class active learning system can be adapted to minimize the negative impact of label noise. We construct a graphical representation of the unlabeled data to encode these relationships and obtain new beliefs on the graph when noisy labels are available. This is demonstrated in three different applications: scene classification, activity classification, and document classification.
  arXiv  Detail & Related papers  (2020-10-18T18:59:44Z)
• Label Noise Types and Their Effects on Deep Learning [0.0]
  In this work, we provide a detailed analysis of the effects of different kinds of label noise on learning. We propose a generic framework to generate feature-dependent label noise, which we show to be the most challenging case for learning. For the ease of other researchers to test their algorithms with noisy labels, we share corrupted labels for the most commonly used benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-23T18:03:39Z)
• Multi-Class Classification from Noisy-Similarity-Labeled Data [98.13491369929798]
  We propose a method for learning from only noisy-similarity-labeled data. We use a noise transition matrix to bridge the class-posterior probability between clean and noisy data. We build a novel learning system which can assign noise-free class labels for instances.
  arXiv  Detail & Related papers  (2020-02-16T05:10:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.