Semi-supervised Learning using Robust Loss
- URL: http://arxiv.org/abs/2203.01524v1
- Date: Thu, 3 Mar 2022 05:34:32 GMT
- Title: Semi-supervised Learning using Robust Loss
- Authors: Wenhui Cui, Haleh Akrami, Anand A. Joshi, Richard M. Leahy
- Abstract summary: We suggest a semi-supervised training strategy for leveraging both manually labeled data and extra unlabeled data.
In contrast to the existing approaches, we apply robust loss for the automated labeled data to compensate for the uneven data quality.
We show that our proposed strategy improves the model performance by compensating for the uneven quality of labels in image classification.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: The amount of manually labeled data is limited in medical applications, so
semi-supervised learning and automatic labeling strategies can be an asset for
training deep neural networks. However, the quality of the automatically
generated labels can be uneven and inferior to manual labels. In this paper, we
suggest a semi-supervised training strategy for leveraging both manually
labeled data and extra unlabeled data. In contrast to the existing approaches,
we apply robust loss for the automated labeled data to automatically compensate
for the uneven data quality using a teacher-student framework. First, we
generate pseudo-labels for unlabeled data using a teacher model pre-trained on
labeled data. These pseudo-labels are noisy, and using them along with labeled
data for training a deep neural network can severely degrade learned feature
representations and the generalization of the network. Here we mitigate the
effect of these pseudo-labels by using robust loss functions. Specifically, we
use three robust loss functions, namely beta cross-entropy, symmetric
cross-entropy, and generalized cross-entropy. We show that our proposed
strategy improves the model performance by compensating for the uneven quality
of labels in image classification as well as segmentation applications.
Related papers
- ERASE: Error-Resilient Representation Learning on Graphs for Label Noise
Tolerance [53.73316938815873]
We propose a method called ERASE (Error-Resilient representation learning on graphs for lAbel noiSe tolerancE) to learn representations with error tolerance.
ERASE combines prototype pseudo-labels with propagated denoised labels and updates representations with error resilience.
Our method can outperform multiple baselines with clear margins in broad noise levels and enjoy great scalability.
arXiv Detail & Related papers (2023-12-13T17:59:07Z) - Soft Curriculum for Learning Conditional GANs with Noisy-Labeled and
Uncurated Unlabeled Data [70.25049762295193]
We introduce a novel conditional image generation framework that accepts noisy-labeled and uncurated data during training.
We propose soft curriculum learning, which assigns instance-wise weights for adversarial training while assigning new labels for unlabeled data.
Our experiments show that our approach outperforms existing semi-supervised and label-noise robust methods in terms of both quantitative and qualitative performance.
arXiv Detail & Related papers (2023-07-17T08:31:59Z) - Doubly Robust Self-Training [46.168395767948965]
We introduce doubly robust self-training, a novel semi-supervised algorithm.
We demonstrate the superiority of the doubly robust loss over the standard self-training baseline.
arXiv Detail & Related papers (2023-06-01T00:57:16Z) - All Points Matter: Entropy-Regularized Distribution Alignment for
Weakly-supervised 3D Segmentation [67.30502812804271]
Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning.
We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
arXiv Detail & Related papers (2023-05-25T08:19:31Z) - Losses over Labels: Weakly Supervised Learning via Direct Loss
Construction [71.11337906077483]
Programmable weak supervision is a growing paradigm within machine learning.
We propose Losses over Labels (LoL) as it creates losses directly from ofs without going through the intermediate step of a label.
We show that LoL improves upon existing weak supervision methods on several benchmark text and image classification tasks.
arXiv Detail & Related papers (2022-12-13T22:29:14Z) - Pseudo-Label Noise Suppression Techniques for Semi-Supervised Semantic
Segmentation [21.163070161951868]
Semi-consuming learning (SSL) can reduce the need for large labelled datasets by incorporating unsupervised data into the training.
Current SSL approaches use an initially supervised trained model to generate predictions for unlabelled images, called pseudo-labels.
We use three mechanisms to control pseudo-label noise and errors.
arXiv Detail & Related papers (2022-10-19T09:46:27Z) - Weighted Distillation with Unlabeled Examples [15.825078347452024]
Distillation with unlabeled examples is a popular and powerful method for training deep neural networks in settings where the amount of labeled data is limited.
This paper proposes a principled approach for addressing this issue based on a ''debiasing'' reweighting of the student's loss function tailored to the distillation training paradigm.
arXiv Detail & Related papers (2022-10-13T04:08:56Z) - Debiased Pseudo Labeling in Self-Training [77.83549261035277]
Deep neural networks achieve remarkable performances on a wide range of tasks with the aid of large-scale labeled datasets.
To mitigate the requirement for labeled data, self-training is widely used in both academia and industry by pseudo labeling on readily-available unlabeled data.
We propose Debiased, in which the generation and utilization of pseudo labels are decoupled by two independent heads.
arXiv Detail & Related papers (2022-02-15T02:14:33Z) - Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
Self-Tuning is a novel approach to enable data-efficient deep learning.
It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model.
It outperforms its SSL and TL counterparts on five tasks by sharp margins.
arXiv Detail & Related papers (2021-02-25T14:56:19Z) - Iterative Label Improvement: Robust Training by Confidence Based
Filtering and Dataset Partitioning [5.1293809610257775]
State-of-the-art, high capacity deep neural networks require large amounts of labelled training data.
They are also highly susceptible to label errors in this data.
We propose a novel meta training and labelling scheme that is able to use inexpensive unlabelled data.
arXiv Detail & Related papers (2020-02-07T10:42:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.