Semi-Supervised Learning using Siamese Networks

• URL: http://arxiv.org/abs/2109.00794v1
• Date: Thu, 2 Sep 2021 09:06:35 GMT
• Title: Semi-Supervised Learning using Siamese Networks
• Authors: Attaullah Sahito, Eibe Frank, and Bernhard Pfahringer
• Abstract summary: This work explores a new training method for semi-supervised learning that is based on similarity function learning using a Siamese network. Confident predictions of unlabeled instances are used as true labels for retraining the Siamese network. For improving unlabeled predictions, local learning with global consistency is also evaluated.
• Score: 3.492636597449942
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks have been successfully used as classification models yielding state-of-the-art results when trained on a large number of labeled samples. These models, however, are more difficult to train successfully for semi-supervised problems where small amounts of labeled instances are available along with a large number of unlabeled instances. This work explores a new training method for semi-supervised learning that is based on similarity function learning using a Siamese network to obtain a suitable embedding. The learned representations are discriminative in Euclidean space, and hence can be used for labeling unlabeled instances using a nearest-neighbor classifier. Confident predictions of unlabeled instances are used as true labels for retraining the Siamese network on the expanded training set. This process is applied iteratively. We perform an empirical study of this iterative self-training algorithm. For improving unlabeled predictions, local learning with global consistency [22] is also evaluated.

Related papers

• Regularizing Neural Network Training via Identity-wise Discriminative Feature Suppression [20.89979858757123]
  When the number of training samples is small, or the class labels are noisy, networks tend to memorize patterns specific to individual instances to minimize the training error. This paper explores a remedy by suppressing the network's tendency to rely on instance-specific patterns for empirical error minimisation.
  arXiv  Detail & Related papers  (2022-09-29T05:14:56Z)
• Compare learning: bi-attention network for few-shot learning [6.559037166322981]
  One of the Few-shot learning methods called metric learning addresses this challenge by first learning a deep distance metric to determine whether a pair of images belong to the same category. In this paper, we propose a novel approach named Bi-attention network to compare the instances, which can measure the similarity between embeddings of instances precisely, globally and efficiently.
  arXiv  Detail & Related papers  (2022-03-25T07:39:10Z)
• Self-Training: A Survey [5.772546394254112]
  Semi-supervised algorithms aim to learn prediction functions from a small set of labeled observations and a large set of unlabeled observations. Among the existing techniques, self-training methods have undoubtedly attracted greater attention in recent years. We present self-training methods for binary and multi-class classification; as well as their variants and two related approaches.
  arXiv  Detail & Related papers  (2022-02-24T11:40:44Z)
• GuidedMix-Net: Semi-supervised Semantic Segmentation by Using Labeled Images as Reference [90.5402652758316]
  We propose a novel method for semi-supervised semantic segmentation named GuidedMix-Net. It uses labeled information to guide the learning of unlabeled instances. It achieves competitive segmentation accuracy and significantly improves the mIoU by +7$%$ compared to previous approaches.
  arXiv  Detail & Related papers  (2021-12-28T06:48:03Z)
• Transfer of Pretrained Model Weights Substantially Improves Semi-Supervised Image Classification [3.492636597449942]
  Deep neural networks produce state-of-the-art results when trained on a large number of labeled examples. Deep neural networks tend to overfit when small amounts of labeled examples are used for training. We show that transfer learning always substantially improves the model's accuracy when few labeled examples are available.
  arXiv  Detail & Related papers  (2021-09-02T08:58:34Z)
• Dash: Semi-Supervised Learning with Dynamic Thresholding [72.74339790209531]
  We propose a semi-supervised learning (SSL) approach that uses unlabeled examples to train models. Our proposed approach, Dash, enjoys its adaptivity in terms of unlabeled data selection.
  arXiv  Detail & Related papers  (2021-09-01T23:52:29Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• Semi-supervised dictionary learning with graph regularization and active points [0.19947949439280027]
  We propose a new semi-supervised dictionary learning method based on two pillars. On one hand, we enforce manifold structure preservation from the original data into sparse code space using Locally Linear Embedding. On the other hand, we train a semi-supervised classifier in sparse code space.
  arXiv  Detail & Related papers  (2020-09-13T09:24:51Z)
• Text Classification with Few Examples using Controlled Generalization [58.971750512415134]
  Current practice relies on pre-trained word embeddings to map words unseen in training to similar seen ones. Our alternative begins with sparse pre-trained representations derived from unlabeled parsed corpora. We show that a feed-forward network over these vectors is especially effective in low-data scenarios.
  arXiv  Detail & Related papers  (2020-05-18T06:04:58Z)
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning [111.03364864022261]
  We propose DivideMix, a framework for learning with noisy labels. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-18T06:20:06Z)
• Learning with Out-of-Distribution Data for Audio Classification [60.48251022280506]
  We show that detecting and relabelling certain OOD instances, rather than discarding them, can have a positive effect on learning. The proposed method is shown to improve the performance of convolutional neural networks by a significant margin.
  arXiv  Detail & Related papers  (2020-02-11T21:08:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.