ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learning

• URL: http://arxiv.org/abs/2303.13556v1
• Date: Wed, 22 Mar 2023 23:51:54 GMT
• Title: ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learning
• Authors: Islam Nassar, Munawar Hayat, Ehsan Abbasnejad, Hamid Rezatofighi, Gholamreza Haffari
• Abstract summary: ProtoCon is a novel method for confidence-based pseudo-labeling. Online nature of ProtoCon allows it to utilise the label history of the entire dataset in one training cycle. It delivers significant gains and faster convergence over state-of-the-art datasets.
• Score: 60.57998388590556
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Confidence-based pseudo-labeling is among the dominant approaches in semi-supervised learning (SSL). It relies on including high-confidence predictions made on unlabeled data as additional targets to train the model. We propose ProtoCon, a novel SSL method aimed at the less-explored label-scarce SSL where such methods usually underperform. ProtoCon refines the pseudo-labels by leveraging their nearest neighbours' information. The neighbours are identified as the training proceeds using an online clustering approach operating in an embedding space trained via a prototypical loss to encourage well-formed clusters. The online nature of ProtoCon allows it to utilise the label history of the entire dataset in one training cycle to refine labels in the following cycle without the need to store image embeddings. Hence, it can seamlessly scale to larger datasets at a low cost. Finally, ProtoCon addresses the poor training signal in the initial phase of training (due to fewer confident predictions) by introducing an auxiliary self-supervised loss. It delivers significant gains and faster convergence over state-of-the-art across 5 datasets, including CIFARs, ImageNet and DomainNet.

Related papers

• Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix [59.55173022987071]
  We study the potential of semi-supervised learning for class-agnostic motion prediction. Our framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data. Our method exhibits comparable performance to weakly and some fully supervised methods.
  arXiv  Detail & Related papers  (2023-12-13T09:32:50Z)
• Contrastive Credibility Propagation for Reliable Semi-Supervised Learning [6.014538614447467]
  We propose Contrastive Credibility Propagation (CCP) for deep SSL via iterative transductive pseudo-label refinement. CCP unifies semi-supervised learning and noisy label learning for the goal of reliably outperforming a supervised baseline in any data scenario.
  arXiv  Detail & Related papers  (2022-11-17T23:01:47Z)
• Deep Low-Density Separation for Semi-Supervised Classification [0.0]
  We introduce a novel hybrid method that applies low-density separation to the embedded features. Our approach effectively classifies thousands of unlabeled users from a relatively small number of hand-classified examples.
  arXiv  Detail & Related papers  (2022-05-22T11:00:55Z)
• CLS: Cross Labeling Supervision for Semi-Supervised Learning [9.929229055862491]
  Cross Labeling Supervision ( CLS) is a framework that generalizes the typical pseudo-labeling process. CLS allows the creation of both pseudo and complementary labels to support both positive and negative learning.
  arXiv  Detail & Related papers  (2022-02-17T08:09:40Z)
• Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation [30.688753736660725]
  We present a simple and yet effective Distribution Alignment and Random Sampling (DARS) method to produce unbiased pseudo labels. Our method performs favorably in comparison with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-23T14:45:14Z)
• Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
  Self-Tuning is a novel approach to enable data-efficient deep learning. It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model. It outperforms its SSL and TL counterparts on five tasks by sharp margins.
  arXiv  Detail & Related papers  (2021-02-25T14:56:19Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)
• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning [52.831894583501395]
  Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications. We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN) We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
  arXiv  Detail & Related papers  (2021-01-02T09:04:14Z)
• Matching Distributions via Optimal Transport for Semi-Supervised Learning [31.533832244923843]
  Semi-Supervised Learning (SSL) approaches have been an influential framework for the usage of unlabeled data. We propose a new approach that adopts an Optimal Transport (OT) technique serving as a metric of similarity between discrete empirical probability measures. We have evaluated our proposed method with state-of-the-art SSL algorithms on standard datasets to demonstrate the superiority and effectiveness of our SSL algorithm.
  arXiv  Detail & Related papers  (2020-12-04T11:15:14Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.