Pseudo-Labeling Based Practical Semi-Supervised Meta-Training for Few-Shot Learning

• URL: http://arxiv.org/abs/2207.06817v3
• Date: Sun, 15 Sep 2024 02:57:43 GMT
• Title: Pseudo-Labeling Based Practical Semi-Supervised Meta-Training for Few-Shot Learning
• Authors: Xingping Dong, Tianran Ouyang, Shengcai Liao, Bo Du, Ling Shao,
• Abstract summary: We propose a simple and effective meta-training framework, called pseudo-labeling based meta-learning (PLML) Firstly, we train a classifier via common semi-supervised learning (SSL) and use it to obtain the pseudo-labels of unlabeled data. We build few-shot tasks from labeled and pseudo-labeled data and design a novel finetuning method with feature smoothing and noise suppression.
• Score: 93.63638405586354
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Most existing few-shot learning (FSL) methods require a large amount of labeled data in meta-training, which is a major limit. To reduce the requirement of labels, a semi-supervised meta-training (SSMT) setting has been proposed for FSL, which includes only a few labeled samples and numbers of unlabeled samples in base classes. However, existing methods under this setting require class-aware sample selection from the unlabeled set, which violates the assumption of unlabeled set. In this paper, we propose a practical semi-supervised meta-training setting with truly unlabeled data to facilitate the applications of FSL in realistic scenarios. To better utilize both the labeled and truly unlabeled data, we propose a simple and effective meta-training framework, called pseudo-labeling based meta-learning (PLML). Firstly, we train a classifier via common semi-supervised learning (SSL) and use it to obtain the pseudo-labels of unlabeled data. Then we build few-shot tasks from labeled and pseudo-labeled data and design a novel finetuning method with feature smoothing and noise suppression to better learn the FSL model from noise labels. Surprisingly, through extensive experiments across two FSL datasets, we find that this simple meta-training framework effectively prevents the performance degradation of various FSL models under limited labeled data, and also significantly outperforms the state-of-the-art SSMT models. Besides, benefiting from meta-training, our method also improves two representative SSL algorithms as well.

Related papers

• Prompt-based Pseudo-labeling Strategy for Sample-Efficient Semi-Supervised Extractive Summarization [12.582774521907227]
  Semi-supervised learning (SSL) is a widely used technique in scenarios where labeled data is scarce and unlabeled data is abundant. Standard SSL methods follow a teacher-student paradigm to first train a classification model and then use the classifier's confidence values to select pseudo-labels. We propose a prompt-based pseudo-labeling strategy with LLMs that picks unlabeled examples with more accurate pseudo-labels.
  arXiv  Detail & Related papers  (2023-11-16T04:29:41Z)
• Generative Semi-supervised Learning with Meta-Optimized Synthetic Samples [5.384630221560811]
  Semi-supervised learning (SSL) is a promising approach for training deep classification models using labeled and unlabeled datasets. In this paper, we investigate the research question: Can we train SSL models without real unlabeled datasets? We propose an SSL method using synthetic datasets generated from generative foundation models trained on datasets containing millions of samples in diverse domains.
  arXiv  Detail & Related papers  (2023-09-28T03:47:26Z)
• DoubleMatch: Improving Semi-Supervised Learning with Self-Supervision [16.757456364034798]
  Semi-supervised learning (SSL) is becoming increasingly popular. We propose a new SSL algorithm, DoubleMatch, which combines the pseudo-labeling technique with a self-supervised loss. We show that this method achieves state-of-the-art accuracies on multiple benchmark datasets while also reducing training times compared to existing SSL methods.
  arXiv  Detail & Related papers  (2022-05-11T15:43:48Z)
• DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples [32.804647684320216]
  Semi-supervised learning (SSL) provides a promising way to leverage unlabeled data by pseudo labels. When the size of labeled data is very small, SSL performs poorly and unstably, possibly due to the low quality of learned pseudo labels. We propose DP-SSL that adopts an innovative data programming scheme to generate probabilistic labels for unlabeled data.
  arXiv  Detail & Related papers  (2021-10-26T14:43:12Z)
• A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning [54.617688468341704]
  Few-shot learning aims to learn models that generalize to novel classes with limited training samples. We propose a novel paradigm containing two parts: (1) a well-designed meta-training algorithm for mitigating ambiguity between base and novel classes caused by unreliable pseudo labels and (2) a model adaptation mechanism to learn discriminative features for novel classes while preserving base knowledge using few labeled and all the unlabeled data.
  arXiv  Detail & Related papers  (2021-10-21T13:25:52Z)
• Dash: Semi-Supervised Learning with Dynamic Thresholding [72.74339790209531]
  We propose a semi-supervised learning (SSL) approach that uses unlabeled examples to train models. Our proposed approach, Dash, enjoys its adaptivity in terms of unlabeled data selection.
  arXiv  Detail & Related papers  (2021-09-01T23:52:29Z)
• The Role of Global Labels in Few-Shot Classification and How to Infer Them [55.64429518100676]
  Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks. We propose Meta Label Learning (MeLa), a novel algorithm that infers global labels and obtains robust few-shot models via standard classification.
  arXiv  Detail & Related papers  (2021-08-09T14:07:46Z)
• Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
  Self-Tuning is a novel approach to enable data-efficient deep learning. It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model. It outperforms its SSL and TL counterparts on five tasks by sharp margins.
  arXiv  Detail & Related papers  (2021-02-25T14:56:19Z)
• TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classification [50.358839666165764]
  We show that the Task-Adaptive Feature Sub-Space Learning (TAFSSL) can significantly boost the performance in Few-Shot Learning scenarios. Specifically, we show that on the challenging miniImageNet and tieredImageNet benchmarks, TAFSSL can improve the current state-of-the-art in both transductive and semi-supervised FSL settings by more than $5%$.
  arXiv  Detail & Related papers  (2020-03-14T16:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.