SLADE: A Self-Training Framework For Distance Metric Learning

• URL: http://arxiv.org/abs/2011.10269v2
• Date: Mon, 29 Mar 2021 20:15:39 GMT
• Title: SLADE: A Self-Training Framework For Distance Metric Learning
• Authors: Jiali Duan, Yen-Liang Lin, Son Tran, Larry S. Davis and C.-C. Jay Kuo
• Abstract summary: We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings.
• Score: 75.54078592084217
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most existing distance metric learning approaches use fully labeled data to learn the sample similarities in an embedding space. We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings. We use self-supervised representation learning to initialize the teacher model. To better deal with noisy pseudo labels generated by the teacher network, we design a new feature basis learning component for the student network, which learns basis functions of feature representations for unlabeled data. The learned basis vectors better measure the pairwise similarity and are used to select high-confident samples for training the student network. We evaluate our method on standard retrieval benchmarks: CUB-200, Cars-196 and In-shop. Experimental results demonstrate that our approach significantly improves the performance over the state-of-the-art methods.

Related papers

• Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation [17.50773984154023]
  We propose a new multi-teacher distillation method based on bi-level optimization (namely BO-NNC) to conduct noisy node classification on the graph data. Specifically, we first employ multiple self-supervised learning methods to train diverse teacher models, and then aggregate their predictions through a teacher weight matrix. Furthermore, we design a new bi-level optimization strategy to dynamically adjust the teacher weight matrix based on the training progress of the student model.
  arXiv  Detail & Related papers  (2024-04-27T12:19:08Z)
• A Benchmark Generative Probabilistic Model for Weak Supervised Learning [2.0257616108612373]
  Weak Supervised Learning approaches have been developed to alleviate the annotation burden. We show that latent variable models (PLVMs) achieve state-of-the-art performance across four datasets.
  arXiv  Detail & Related papers  (2023-03-31T07:06:24Z)
• Q-Match: Self-supervised Learning by Matching Distributions Induced by a Queue [6.1678491628787455]
  We introduce our algorithm, Q-Match, and show it is possible to induce the student-teacher distributions without any knowledge of downstream classes. We show that our method is sample efficient--in terms of both the labels required for downstream training and the amount of unlabeled data required for pre-training--and scales well to the sizes of both the labeled and unlabeled data.
  arXiv  Detail & Related papers  (2023-02-10T18:59:05Z)
• Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant [72.4512562104361]
  We argue that the unlabeled data with pseudo labels can facilitate the learning of representative features in the feature extractor. Motivated by this consideration, we propose a novel framework, Gentle Teaching Assistant (GTA-Seg) to disentangle the effects of pseudo labels on feature extractor and mask predictor.
  arXiv  Detail & Related papers  (2023-01-18T07:11:24Z)
• Online pseudo labeling for polyp segmentation with momentum networks [5.920947681019466]
  In semi-supervised learning, the quality of labels plays a crucial role in model performance. We present a new pseudo labeling strategy that enhances the quality of pseudo labels used for training student networks. Our results surpass common practice by 3% and even approach fully-supervised results on some datasets.
  arXiv  Detail & Related papers  (2022-09-29T07:33:54Z)
• Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition [98.25592165484737]
  We propose a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL) CMPL achieves $17.6%$ and $25.1%$ Top-1 accuracy on Kinetics-400 and UCF-101 using only the RGB modality and $1%$ labeled data, respectively.
  arXiv  Detail & Related papers  (2021-12-17T18:59:41Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)
• Improving Semantic Segmentation via Self-Training [75.07114899941095]
  We show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm. We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data. Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets.
  arXiv  Detail & Related papers  (2020-04-30T17:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.