Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation

• URL: http://arxiv.org/abs/2405.20596v1
• Date: Fri, 31 May 2024 03:13:45 GMT
• Title: Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation
• Authors: Jiachen Liang, Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, Xilin Chen,
• Abstract summary: Traditional semi-supervised learning assumes that the feature distributions of labeled and unlabeled data are consistent. We propose Self-Supervised Feature Adaptation (SSFA), a generic framework for improving SSL performance when labeled and unlabeled data come from different distributions. Our proposed SSFA is applicable to various pseudo-label-based SSL learners and significantly improves performance in labeled, unlabeled, and even unseen distributions.
• Score: 87.17768598044427
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traditional semi-supervised learning (SSL) assumes that the feature distributions of labeled and unlabeled data are consistent which rarely holds in realistic scenarios. In this paper, we propose a novel SSL setting, where unlabeled samples are drawn from a mixed distribution that deviates from the feature distribution of labeled samples. Under this setting, previous SSL methods tend to predict wrong pseudo-labels with the model fitted on labeled data, resulting in noise accumulation. To tackle this issue, we propose Self-Supervised Feature Adaptation (SSFA), a generic framework for improving SSL performance when labeled and unlabeled data come from different distributions. SSFA decouples the prediction of pseudo-labels from the current model to improve the quality of pseudo-labels. Particularly, SSFA incorporates a self-supervised task into the SSL framework and uses it to adapt the feature extractor of the model to the unlabeled data. In this way, the extracted features better fit the distribution of unlabeled data, thereby generating high-quality pseudo-labels. Extensive experiments show that our proposed SSFA is applicable to various pseudo-label-based SSL learners and significantly improves performance in labeled, unlabeled, and even unseen distributions.

Related papers

• Learning Label Refinement and Threshold Adjustment for Imbalanced Semi-Supervised Learning [6.904448748214652]
  Semi-supervised learning algorithms struggle to perform well when exposed to imbalanced training data. We introduce SEmi-supervised learning with pseudo-label optimization based on VALidation data (SEVAL) SEVAL adapts to specific tasks with improved pseudo-labels accuracy and ensures pseudo-labels correctness on a per-class basis.
  arXiv  Detail & Related papers  (2024-07-07T13:46:22Z)
• InstanT: Semi-supervised Learning with Instance-dependent Thresholds [75.91684890150283]
  We propose the study of instance-dependent thresholds, which has the highest degree of freedom compared with existing methods. We devise a novel instance-dependent threshold function for all unlabeled instances by utilizing their instance-level ambiguity and the instance-dependent error rates of pseudo-labels.
  arXiv  Detail & Related papers  (2023-10-29T05:31:43Z)
• NorMatch: Matching Normalizing Flows with Discriminative Classifiers for Semi-Supervised Learning [8.749830466953584]
  Semi-Supervised Learning (SSL) aims to learn a model using a tiny labeled set and massive amounts of unlabeled data. In this work we introduce a new framework for SSL named NorMatch. We demonstrate, through numerical and visual results, that NorMatch achieves state-of-the-art performance on several datasets.
  arXiv  Detail & Related papers  (2022-11-17T15:39:18Z)
• Complementing Semi-Supervised Learning with Uncertainty Quantification [6.612035830987296]
  We propose a novel unsupervised uncertainty-aware objective that relies on aleatoric and epistemic uncertainty quantification. Our results outperform the state-of-the-art results on complex datasets such as CIFAR-100 and Mini-ImageNet.
  arXiv  Detail & Related papers  (2022-07-22T00:15:02Z)
• DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples [32.804647684320216]
  Semi-supervised learning (SSL) provides a promising way to leverage unlabeled data by pseudo labels. When the size of labeled data is very small, SSL performs poorly and unstably, possibly due to the low quality of learned pseudo labels. We propose DP-SSL that adopts an innovative data programming scheme to generate probabilistic labels for unlabeled data.
  arXiv  Detail & Related papers  (2021-10-26T14:43:12Z)
• Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning [80.05441565830726]
  This paper addresses imbalanced semi-supervised learning, where heavily biased pseudo-labels can harm the model performance. We propose a general pseudo-labeling framework to address the bias motivated by this observation. We term the novel pseudo-labeling framework for imbalanced SSL as Distribution-Aware Semantics-Oriented (DASO) Pseudo-label.
  arXiv  Detail & Related papers  (2021-06-10T11:58:25Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning [126.31716228319902]
  We develop Distribution Aligning Refinery of Pseudo-label (DARP) algorithm. We show that DARP is provably and efficiently compatible with state-of-the-art SSL schemes.
  arXiv  Detail & Related papers  (2020-07-17T09:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.