Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models

• URL: http://arxiv.org/abs/2006.10236v1
• Date: Thu, 18 Jun 2020 02:10:56 GMT
• Title: Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models
• Authors: Siavash Khodadadeh, Sharare Zehtabian, Saeed Vahidian, Weijia Wang, Bill Lin and Ladislau B\"ol\"oni
• Abstract summary: We describe an approach that generates meta-tasks using generative models. We find that the proposed approach, LAtent Space Interpolation Unsupervised Meta-learning (LASIUM), outperforms or is competitive with current unsupervised learning baselines.
• Score: 11.943374020641214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised meta-learning approaches rely on synthetic meta-tasks that are created using techniques such as random selection, clustering and/or augmentation. Unfortunately, clustering and augmentation are domain-dependent, and thus they require either manual tweaking or expensive learning. In this work, we describe an approach that generates meta-tasks using generative models. A critical component is a novel approach of sampling from the latent space that generates objects grouped into synthetic classes forming the training and validation data of a meta-task. We find that the proposed approach, LAtent Space Interpolation Unsupervised Meta-learning (LASIUM), outperforms or is competitive with current unsupervised learning baselines on few-shot classification tasks on the most widely used benchmark datasets. In addition, the approach promises to be applicable without manual tweaking over a wider range of domains than previous approaches.

Related papers

• Unified Generative and Discriminative Training for Multi-modal Large Language Models [88.84491005030316]
  Generative training has enabled Vision-Language Models (VLMs) to tackle various complex tasks. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval. This paper proposes a unified approach that integrates the strengths of both paradigms.
  arXiv  Detail & Related papers  (2024-11-01T01:51:31Z)
• Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis [53.38518232934096]
  Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance. We propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features. In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is motivated by applications to Earth science.
  arXiv  Detail & Related papers  (2024-06-12T08:30:16Z)
• Unsupervised Meta-Learning via In-Context Learning [3.4165401459803335]
  We propose a novel approach to unsupervised meta-learning that leverages the generalization abilities of in-supervised learning. Our method reframes meta-learning as a sequence modeling problem, enabling the transformer encoder to learn task context from support images.
  arXiv  Detail & Related papers  (2024-05-25T08:29:46Z)
• Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterances [24.142013877384603]
  This paper introduces a novel unsupervised multimodal clustering method (UMC), making a pioneering contribution to this field. UMC introduces a unique approach to constructing augmentation views for multimodal data, which are then used to perform pre-training. We show remarkable improvements of 2-6% scores in clustering metrics over state-of-the-art methods, marking the first successful endeavor in this domain.
  arXiv  Detail & Related papers  (2024-05-21T13:24:07Z)
• Unsupervised Meta-Learning via Few-shot Pseudo-supervised Contrastive Learning [72.3506897990639]
  We propose a simple yet effective unsupervised meta-learning framework, coined Pseudo-supervised Contrast (PsCo) for few-shot classification. PsCo outperforms existing unsupervised meta-learning methods under various in-domain and cross-domain few-shot classification benchmarks.
  arXiv  Detail & Related papers  (2023-03-02T06:10:13Z)
• Meta-Learning via Classifier(-free) Guidance [5.812784742024491]
  State-of-the-art meta-learning techniques do not optimize for zero-shot adaptation to unseen tasks. We propose meta-learning techniques that use natural language guidance to achieve higher zero-shot performance.
  arXiv  Detail & Related papers  (2022-10-17T11:09:35Z)
• Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning [146.11600461034746]
  Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
  arXiv  Detail & Related papers  (2022-09-27T19:04:36Z)
• Set-based Meta-Interpolation for Few-Task Meta-Learning [79.4236527774689]
  We propose a novel domain-agnostic task augmentation method, Meta-Interpolation, to densify the meta-training task distribution. We empirically validate the efficacy of Meta-Interpolation on eight datasets spanning across various domains.
  arXiv  Detail & Related papers  (2022-05-20T06:53:03Z)
• Double Meta-Learning for Data Efficient Policy Optimization in Non-Stationary Environments [12.45281856559346]
  We are interested in learning models of non-stationary environments, which can be framed as a multi-task learning problem. Model-free reinforcement learning algorithms can achieve good performance in multi-task learning at a cost of extensive sampling. While model-based approaches are among the most data efficient learning algorithms, they still struggle with complex tasks and model uncertainties.
  arXiv  Detail & Related papers  (2020-11-21T03:19:35Z)
• Information-Theoretic Generalization Bounds for Meta-Learning and Applications [42.275148861039895]
  Key performance measure for meta-learning is the meta-generalization gap. This paper presents novel information-theoretic upper bounds on the meta-generalization gap.
  arXiv  Detail & Related papers  (2020-05-09T05:48:01Z)
• Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
  We develop a notion of incremental learning during the meta-training phase of meta-learning. Our approach performs favorably at test time as compared to training a model with the full meta-training set.
  arXiv  Detail & Related papers  (2020-02-11T01:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.