Boosting the Performance of Semi-Supervised Learning with Unsupervised Clustering

• URL: http://arxiv.org/abs/2012.00504v1
• Date: Tue, 1 Dec 2020 14:19:14 GMT
• Title: Boosting the Performance of Semi-Supervised Learning with Unsupervised Clustering
• Authors: Boaz Lerner, Guy Shiran, Daphna Weinshall
• Abstract summary: We show that ignoring labels altogether for whole epochs intermittently during training can significantly improve performance in the small sample regime. We demonstrate our method's efficacy in boosting several state-of-the-art SSL algorithms.
• Score: 10.033658645311188
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Semi-Supervised Learning (SSL) has shown much promise in leveraging unlabeled data while being provided with very few labels. In this paper, we show that ignoring the labels altogether for whole epochs intermittently during training can significantly improve performance in the small sample regime. More specifically, we propose to train a network on two tasks jointly. The primary classification task is exposed to both the unlabeled and the scarcely annotated data, whereas the secondary task seeks to cluster the data without any labels. As opposed to hand-crafted pretext tasks frequently used in self-supervision, our clustering phase utilizes the same classification network and head in an attempt to relax the primary task and propagate the information from the labels without overfitting them. On top of that, the self-supervised technique of classifying image rotations is incorporated during the unsupervised learning phase to stabilize training. We demonstrate our method's efficacy in boosting several state-of-the-art SSL algorithms, significantly improving their results and reducing running time in various standard semi-supervised benchmarks, including 92.6% accuracy on CIFAR-10 and 96.9% on SVHN, using only 4 labels per class in each task. We also notably improve the results in the extreme cases of 1,2 and 3 labels per class, and show that features learned by our model are more meaningful for separating the data.

Related papers

• Boosting Consistency in Dual Training for Long-Tailed Semi-Supervised Learning [49.07038093130949]
  Long-tailed semi-supervised learning (LTSSL) algorithms assume that the class distributions of labeled and unlabeled data are almost identical. We propose a new simple method that can effectively utilize unlabeled data from unknown class distributions. We show that BOAT achieves state-of-the-art performance on a variety of standard LTSSL benchmarks.
  arXiv  Detail & Related papers  (2024-06-19T03:35:26Z)
• Incremental Self-training for Semi-supervised Learning [56.57057576885672]
  IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed.
  arXiv  Detail & Related papers  (2024-04-14T05:02:00Z)
• Bridging the Gap: Learning Pace Synchronization for Open-World Semi-Supervised Learning [44.91863420044712]
  In open-world semi-supervised learning, a machine learning model is tasked with uncovering novel categories from unlabeled data. We introduce 1) the adaptive synchronizing marginal loss which imposes class-specific negative margins to alleviate the model bias towards seen classes, and 2) the pseudo-label contrastive clustering which exploits pseudo-labels predicted by the model to group unlabeled data from the same category together. Our method balances the learning pace between seen and novel classes, achieving a remarkable 3% average accuracy increase on the ImageNet dataset.
  arXiv  Detail & Related papers  (2023-09-21T09:44:39Z)
• PercentMatch: Percentile-based Dynamic Thresholding for Multi-Label Semi-Supervised Classification [64.39761523935613]
  We propose a percentile-based threshold adjusting scheme to dynamically alter the score thresholds of positive and negative pseudo-labels for each class during the training. We achieve strong performance on Pascal VOC2007 and MS-COCO datasets when compared to recent SSL methods.
  arXiv  Detail & Related papers  (2022-08-30T01:27:48Z)
• CLS: Cross Labeling Supervision for Semi-Supervised Learning [9.929229055862491]
  Cross Labeling Supervision ( CLS) is a framework that generalizes the typical pseudo-labeling process. CLS allows the creation of both pseudo and complementary labels to support both positive and negative learning.
  arXiv  Detail & Related papers  (2022-02-17T08:09:40Z)
• CaSP: Class-agnostic Semi-Supervised Pretraining for Detection and Segmentation [60.28924281991539]
  We propose a novel Class-agnostic Semi-supervised Pretraining (CaSP) framework to achieve a more favorable task-specificity balance. Using 3.6M unlabeled data, we achieve a remarkable performance gain of 4.7% over ImageNet-pretrained baseline on object detection.
  arXiv  Detail & Related papers  (2021-12-09T14:54:59Z)
• STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification [0.8662293148437356]
  Semi-supervised learning (SSL) has proven to be effective at leveraging large-scale unlabeled data. Recent SSL methods rely on unlabeled image data at a scale of billions to work well. We propose noisy semi-supervised transfer learning, which integrates transfer learning and self-training with noisy student.
  arXiv  Detail & Related papers  (2021-08-18T19:35:05Z)
• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning [52.831894583501395]
  Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications. We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN) We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
  arXiv  Detail & Related papers  (2021-01-02T09:04:14Z)
• Are Fewer Labels Possible for Few-shot Learning? [81.89996465197392]
  Few-shot learning is challenging due to its very limited data and labels. Recent studies in big transfer (BiT) show that few-shot learning can greatly benefit from pretraining on large scale labeled dataset in a different domain. We propose eigen-finetuning to enable fewer shot learning by leveraging the co-evolution of clustering and eigen-samples in the finetuning.
  arXiv  Detail & Related papers  (2020-12-10T18:59:29Z)
• Deep Categorization with Semi-Supervised Self-Organizing Maps [0.0]
  This article presents a semi-supervised model, called Batch Semi-Supervised Self-Organizing Map (Batch SS-SOM) The results show that Batch SS-SOM is a good option for semi-supervised classification and clustering. It performs well in terms of accuracy and clustering error, even with a small number of labeled samples.
  arXiv  Detail & Related papers  (2020-06-17T22:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.