Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning

• URL: http://arxiv.org/abs/2209.13635v1
• Date: Tue, 27 Sep 2022 19:04:36 GMT
• Title: Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning
• Authors: Xingping Dong, Jianbing Shen, and Ling Shao
• Abstract summary: Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
• Score: 146.11600461034746
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The pioneering method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. However, it often suffers from label inconsistency or limited diversity, which leads to poor performance. In this work, we prove that the core reason for this is lack of a clustering-friendly property in the embedding space. We address this by minimizing the inter- to intra-class similarity ratio to provide clustering-friendly embedding features, and validate our approach through comprehensive experiments. Note that, despite only utilizing a simple clustering algorithm (k-means) in our embedding space to obtain the pseudo-labels, we achieve significant improvement. Moreover, we adopt a progressive evaluation mechanism to obtain more diverse samples in order to further alleviate the limited diversity problem. Finally, our approach is also model-agnostic and can easily be integrated into existing supervised methods. To demonstrate its generalization ability, we integrate it into two representative algorithms: MAML and EP. The results on three main few-shot benchmarks clearly show that the proposed method achieves significant improvement compared to state-of-the-art models. Notably, our approach also outperforms the corresponding supervised method in two tasks.

Related papers

• Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning [81.83013974171364]
  Semi-supervised multi-label learning (SSMLL) is a powerful framework for leveraging unlabeled data to reduce the expensive cost of collecting precise multi-label annotations. Unlike semi-supervised learning, one cannot select the most probable label as the pseudo-label in SSMLL due to multiple semantics contained in an instance. We propose a dual-perspective method to generate high-quality pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-26T09:33:53Z)
• GCC: Generative Calibration Clustering [55.44944397168619]
  We propose a novel Generative Clustering (GCC) method to incorporate feature learning and augmentation into clustering procedure. First, we develop a discrimirative feature alignment mechanism to discover intrinsic relationship across real and generated samples. Second, we design a self-supervised metric learning to generate more reliable cluster assignment.
  arXiv  Detail & Related papers  (2024-04-14T01:51:11Z)
• New Intent Discovery with Attracting and Dispersing Prototype [32.607743914487614]
  New Intent Discovery (NID) aims to recognize known and infer new intent categories with the help of limited labeled and large-scale unlabeled data. We propose a Robust and Adaptive Prototypical learning framework for globally distinct decision boundaries for both known and new intent categories.
  arXiv  Detail & Related papers  (2024-03-25T16:31:55Z)
• One-Step Multi-View Clustering Based on Transition Probability [61.841829428397034]
  We introduce the One-Step Multi-View Clustering Based on Transition Probability (OSMVC-TP) Our method directly learns the transition probabilities from anchor points to categories, and calculates the transition probabilities from samples to categories. To maintain consistency in labels across different views, we apply a Schatten p-norm constraint on the tensor composed of the soft labels.
  arXiv  Detail & Related papers  (2024-03-03T09:43:23Z)
• Anchor-free Clustering based on Anchor Graph Factorization [17.218481911995365]
  We introduce a novel method termed Anchor-free Clustering based on Anchor Graph Factorization (AFCAGF) AFCAGF innovates in learning the anchor graph, requiring only the computation of pairwise distances between samples. We evolve the concept of the membership matrix between cluster centers and samples in FKM into an anchor graph encompassing multiple anchor points and samples.
  arXiv  Detail & Related papers  (2024-02-24T02:16:42Z)
• Efficient Bilateral Cross-Modality Cluster Matching for Unsupervised Visible-Infrared Person ReID [56.573905143954015]
  We propose a novel bilateral cluster matching-based learning framework to reduce the modality gap by matching cross-modality clusters. Under such a supervisory signal, a Modality-Specific and Modality-Agnostic (MSMA) contrastive learning framework is proposed to align features jointly at a cluster-level. Experiments on the public SYSU-MM01 and RegDB datasets demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2023-05-22T03:27:46Z)
• Neighbour Consistency Guided Pseudo-Label Refinement for Unsupervised Person Re-Identification [80.98291772215154]
  Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels. We propose a Neighbour Consistency guided Pseudo Label Refinement framework.
  arXiv  Detail & Related papers  (2022-11-30T09:39:57Z)
• Learning the Precise Feature for Cluster Assignment [39.320210567860485]
  We propose a framework which integrates representation learning and clustering into a single pipeline for the first time. The proposed framework exploits the powerful ability of recently developed generative models for learning intrinsic features. Experimental results show that the performance of the proposed method is superior, or at least comparable to, the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-11T04:08:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.