Label Noise: Correcting a Correction
- URL: http://arxiv.org/abs/2307.13100v1
- Date: Mon, 24 Jul 2023 19:41:19 GMT
- Title: Label Noise: Correcting a Correction
- Authors: William Toner, Amos Storkey
- Abstract summary: Training neural network classifiers on datasets with label noise poses a risk of overfitting them to the noisy labels.
We propose a more direct approach to tackling overfitting caused by label noise.
We provide theoretical results that yield explicit, easily computable bounds on the minimum achievable noisy risk for different loss functions.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Training neural network classifiers on datasets with label noise poses a risk
of overfitting them to the noisy labels. To address this issue, researchers
have explored alternative loss functions that aim to be more robust. However,
many of these alternatives are heuristic in nature and still vulnerable to
overfitting or underfitting. In this work, we propose a more direct approach to
tackling overfitting caused by label noise. We observe that the presence of
label noise implies a lower bound on the noisy generalised risk. Building upon
this observation, we propose imposing a lower bound on the empirical risk
during training to mitigate overfitting. Our main contribution is providing
theoretical results that yield explicit, easily computable bounds on the
minimum achievable noisy risk for different loss functions. We empirically
demonstrate that using these bounds significantly enhances robustness in
various settings, with virtually no additional computational cost.
Related papers
- Binary Classification with Instance and Label Dependent Label Noise [4.061135251278187]
We show that learning with noisy samples is impossible without access to clean samples or strong assumptions on the distribution of the data.
Our findings suggest that learning solely with noisy samples is impossible without access to clean samples or strong assumptions on the distribution of the data.
arXiv Detail & Related papers (2023-06-06T04:47:44Z) - Enhancing Contrastive Learning with Noise-Guided Attack: Towards
Continual Relation Extraction in the Wild [57.468184469589744]
We develop a noise-resistant contrastive framework named as textbfNoise-guided textbfattack in textbfContrative textbfLearning(NaCL)
Compared to direct noise discarding or inaccessible noise relabeling, we present modifying the feature space to match the given noisy labels via attacking.
arXiv Detail & Related papers (2023-05-11T18:48:18Z) - Logistic-Normal Likelihoods for Heteroscedastic Label Noise [6.947162493996924]
A natural way of estimating heteroscedastic label noise in regression is to model the observed (potentially noisy) target as a sample from a normal distribution.
We propose an extension of this simple and probabilistic approach to classification that has the same desirable loss attenuation properties.
arXiv Detail & Related papers (2023-04-06T03:45:07Z) - Do We Need to Penalize Variance of Losses for Learning with Label Noise? [91.38888889609002]
We find that the variance should be increased for the problem of learning with noisy labels.
By exploiting the label noise transition matrix, regularizers can be easily designed to reduce the variance of losses.
Empirically, the proposed method by increasing the variance of losses significantly improves the generalization ability of baselines on both synthetic and real-world datasets.
arXiv Detail & Related papers (2022-01-30T06:19:08Z) - S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
In this paper we address the problem of classification in the presence of label noise.
In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space.
Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
arXiv Detail & Related papers (2021-11-22T15:49:20Z) - Robustness and reliability when training with noisy labels [12.688634089849023]
Labelling of data for supervised learning can be costly and time-consuming.
Deep neural networks have proved capable of fitting random labels, regularisation and the use of robust loss functions.
arXiv Detail & Related papers (2021-10-07T10:30:20Z) - Open-set Label Noise Can Improve Robustness Against Inherent Label Noise [27.885927200376386]
We show that open-set noisy labels can be non-toxic and even benefit the robustness against inherent noisy labels.
We propose a simple yet effective regularization by introducing Open-set samples with Dynamic Noisy Labels (ODNL) into training.
arXiv Detail & Related papers (2021-06-21T07:15:50Z) - Tackling Instance-Dependent Label Noise via a Universal Probabilistic
Model [80.91927573604438]
This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances.
Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
arXiv Detail & Related papers (2021-01-14T05:43:51Z) - A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
The presence of label noise often misleads the training of deep neural networks.
We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
arXiv Detail & Related papers (2020-12-22T06:36:58Z) - Improving Face Recognition by Clustering Unlabeled Faces in the Wild [77.48677160252198]
We propose a novel identity separation method based on extreme value theory.
It greatly reduces the problems caused by overlapping-identity label noise.
Experiments on both controlled and real settings demonstrate our method's consistent improvements.
arXiv Detail & Related papers (2020-07-14T12:26:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.