Semi-supervised learning by selective training with pseudo labels via confidence estimation

• URL: http://arxiv.org/abs/2103.08193v1
• Date: Mon, 15 Mar 2021 08:00:33 GMT
• Title: Semi-supervised learning by selective training with pseudo labels via confidence estimation
• Authors: Masato Ishii
• Abstract summary: We propose a novel semi-supervised learning (SSL) method that adopts selective training with pseudo labels. In our method, we generate hard pseudo-labels and also estimate their confidence, which represents how likely each pseudo-label is to be correct. We also propose a new data augmentation method, called MixConf, that enables us to obtain confidence-calibrated models even when the number of training data is small.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel semi-supervised learning (SSL) method that adopts selective training with pseudo labels. In our method, we generate hard pseudo-labels and also estimate their confidence, which represents how likely each pseudo-label is to be correct. Then, we explicitly select which pseudo-labeled data should be used to update the model. Specifically, assuming that loss on incorrectly pseudo-labeled data sensitively increase against data augmentation, we select the data corresponding to relatively small loss after applying data augmentation. The confidence is used not only for screening candidates of pseudo-labeled data to be selected but also for automatically deciding how many pseudo-labeled data should be selected within a mini-batch. Since accurate estimation of the confidence is crucial in our method, we also propose a new data augmentation method, called MixConf, that enables us to obtain confidence-calibrated models even when the number of training data is small. Experimental results with several benchmark datasets validate the advantage of our SSL method as well as MixConf.

Related papers

• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning [73.13448439554497]
  Semi-Supervised Learning (SSL) has been an effective way to leverage abundant unlabeled data with extremely scarce labeled data. Most SSL methods are commonly based on instance-wise consistency between different data transformations. We propose FlatMatch which minimizes a cross-sharpness measure to ensure consistent learning performance between the two datasets.
  arXiv  Detail & Related papers  (2023-10-25T06:57:59Z)
• Boosting Semi-Supervised Learning by bridging high and low-confidence predictions [4.18804572788063]
  Pseudo-labeling is a crucial technique in semi-supervised learning (SSL) We propose a new method called ReFixMatch, which aims to utilize all of the unlabeled data during training.
  arXiv  Detail & Related papers  (2023-08-15T00:27:18Z)
• Doubly Robust Self-Training [46.168395767948965]
  We introduce doubly robust self-training, a novel semi-supervised algorithm. We demonstrate the superiority of the doubly robust loss over the standard self-training baseline.
  arXiv  Detail & Related papers  (2023-06-01T00:57:16Z)
• InPL: Pseudo-labeling the Inliers First for Imbalanced Semi-supervised Learning [34.062061310242385]
  We present a new perspective of pseudo-labeling for imbalanced semi-supervised learning (SSL) We measure whether an unlabeled sample is likely to be in-distribution'' or out-of-distribution'' Experiments demonstrate that our energy-based pseudo-labeling method, textbfInPL, significantly outperforms confidence-based methods on imbalanced SSL benchmarks.
  arXiv  Detail & Related papers  (2023-03-13T16:45:41Z)
• Pseudo-Label Noise Suppression Techniques for Semi-Supervised Semantic Segmentation [21.163070161951868]
  Semi-consuming learning (SSL) can reduce the need for large labelled datasets by incorporating unsupervised data into the training. Current SSL approaches use an initially supervised trained model to generate predictions for unlabelled images, called pseudo-labels. We use three mechanisms to control pseudo-label noise and errors.
  arXiv  Detail & Related papers  (2022-10-19T09:46:27Z)
• Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization [88.74813798138466]
  Localizing keypoints of an object is a basic visual problem. Supervised learning of a keypoint localization network often requires a large amount of data. We propose to automatically select reliable pseudo-labeled samples with a series of dynamic thresholds.
  arXiv  Detail & Related papers  (2022-01-21T09:51:58Z)
• Boosting Semi-Supervised Face Recognition with Noise Robustness [54.342992887966616]
  This paper presents an effective solution to semi-supervised face recognition that is robust to the label noise aroused by the auto-labelling. We develop a semi-supervised face recognition solution, named Noise Robust Learning-Labelling (NRoLL), which is based on the robust training ability empowered by GN.
  arXiv  Detail & Related papers  (2021-05-10T14:43:11Z)
• Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
  Self-Tuning is a novel approach to enable data-efficient deep learning. It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model. It outperforms its SSL and TL counterparts on five tasks by sharp margins.
  arXiv  Detail & Related papers  (2021-02-25T14:56:19Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.