Robust Semi-supervised Learning by Wisely Leveraging Open-set Data

• URL: http://arxiv.org/abs/2405.06979v2
• Date: Mon, 20 May 2024 08:37:48 GMT
• Title: Robust Semi-supervised Learning by Wisely Leveraging Open-set Data
• Authors: Yang Yang, Nan Jiang, Yi Xu, De-Chuan Zhan,
• Abstract summary: Open-set Semi-supervised Learning (OSSL) holds a realistic setting that unlabeled data may come from classes unseen in the labeled set. We propose Wise Open-set Semi-supervised Learning (WiseOpen), a generic OSSL framework that selectively leverages the open-set data for training the model.
• Score: 48.67897991121204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-set Semi-supervised Learning (OSSL) holds a realistic setting that unlabeled data may come from classes unseen in the labeled set, i.e., out-of-distribution (OOD) data, which could cause performance degradation in conventional SSL models. To handle this issue, except for the traditional in-distribution (ID) classifier, some existing OSSL approaches employ an extra OOD detection module to avoid the potential negative impact of the OOD data. Nevertheless, these approaches typically employ the entire set of open-set data during their training process, which may contain data unfriendly to the OSSL task that can negatively influence the model performance. This inspires us to develop a robust open-set data selection strategy for OSSL. Through a theoretical understanding from the perspective of learning theory, we propose Wise Open-set Semi-supervised Learning (WiseOpen), a generic OSSL framework that selectively leverages the open-set data for training the model. By applying a gradient-variance-based selection mechanism, WiseOpen exploits a friendly subset instead of the whole open-set dataset to enhance the model's capability of ID classification. Moreover, to reduce the computational expense, we also propose two practical variants of WiseOpen by adopting low-frequency update and loss-based selection respectively. Extensive experiments demonstrate the effectiveness of WiseOpen in comparison with the state-of-the-art.

Related papers

• SCOMatch: Alleviating Overtrusting in Open-set Semi-supervised Learning [25.508200663171625]
  Open-set semi-supervised learning (OSSL) uses practical open-set unlabeled data. Prior OSSL methods suffer from the tendency to overtrust the labeled ID data. We propose SCOMatch, a novel OSSL that treats OOD samples as an additional class, forming a new SSL process.
  arXiv  Detail & Related papers  (2024-09-26T03:47:34Z)
• SSB: Simple but Strong Baseline for Boosting Performance of Open-Set Semi-Supervised Learning [106.46648817126984]
  In this paper, we study the challenging and realistic open-set SSL setting. The goal is to both correctly classify inliers and to detect outliers. We find that inlier classification performance can be largely improved by incorporating high-confidence pseudo-labeled data.
  arXiv  Detail & Related papers  (2023-11-17T15:14:40Z)
• Progressive Feature Adjustment for Semi-supervised Learning from Pretrained Models [39.42802115580677]
  Semi-supervised learning (SSL) can leverage both labeled and unlabeled data to build a predictive model. Recent literature suggests that naively applying state-of-the-art SSL with a pretrained model fails to unleash the full potential of training data. We propose to use pseudo-labels from the unlabelled data to update the feature extractor that is less sensitive to incorrect labels.
  arXiv  Detail & Related papers  (2023-09-09T01:57:14Z)
• Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning [69.81438976273866]
  Open-set semi-supervised learning (Open-set SSL) considers a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers) We introduce evidential deep learning (EDL) as an outlier detector to quantify different types of uncertainty, and design different uncertainty metrics for self-training and inference. We propose a novel adaptive negative optimization strategy, making EDL more tailored to the unlabeled dataset containing both inliers and outliers.
  arXiv  Detail & Related papers  (2023-03-21T09:07:15Z)
• Improving Open-Set Semi-Supervised Learning with Self-Supervision [13.944469874692459]
  Open-set semi-supervised learning (OSSL) embodies a practical scenario within semi-supervised learning. We propose an OSSL framework that facilitates learning from all unlabeled data through self-supervision. Our method yields state-of-the-art results on many of the evaluated benchmark problems.
  arXiv  Detail & Related papers  (2023-01-24T16:46:37Z)
• Prompt-driven efficient Open-set Semi-supervised Learning [52.30303262499391]
  Open-set semi-supervised learning (OSSL) has attracted growing interest, which investigates a more practical scenario where out-of-distribution (OOD) samples are only contained in unlabeled data. We propose a prompt-driven efficient OSSL framework, called OpenPrompt, which can propagate class information from labeled to unlabeled data with only a small number of trainable parameters.
  arXiv  Detail & Related papers  (2022-09-28T16:25:08Z)
• Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning [101.28281124670647]
  Open-set semi-supervised learning (open-set SSL) investigates a challenging but practical scenario where out-of-distribution (OOD) samples are contained in the unlabeled data. We propose a novel training mechanism that could effectively exploit the presence of OOD data for enhanced feature learning. Our approach substantially lifts the performance on open-set SSL and outperforms the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2021-08-12T09:14:44Z)
• OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers [71.08167292329028]
  We propose a novel Open-set Semi-Supervised Learning (OSSL) approach called OpenMatch. OpenMatch unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers. It achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
  arXiv  Detail & Related papers  (2021-05-28T23:57:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.