Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks

• URL: http://arxiv.org/abs/2002.06753v3
• Date: Wed, 1 Jul 2020 13:59:50 GMT
• Title: Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks
• Authors: Micah Goldblum, Steven Reich, Liam Fowl, Renkun Ni, Valeriia Cherepanova, Tom Goldstein
• Abstract summary: We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models trained classically. We develop a regularizer which boosts the performance of standard training routines for few-shot classification.
• Score: 55.66438591090072
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Meta-learning algorithms produce feature extractors which achieve state-of-the-art performance on few-shot classification. While the literature is rich with meta-learning methods, little is known about why the resulting feature extractors perform so well. We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models which are trained classically. In doing so, we introduce and verify several hypotheses for why meta-learned models perform better. Furthermore, we develop a regularizer which boosts the performance of standard training routines for few-shot classification. In many cases, our routine outperforms meta-learning while simultaneously running an order of magnitude faster.

Related papers

• Meta-Learning with Self-Improving Momentum Target [72.98879709228981]
  We propose Self-improving Momentum Target (SiMT) to improve the performance of a meta-learner. SiMT generates the target model by adapting from the temporal ensemble of the meta-learner. We show that SiMT brings a significant performance gain when combined with a wide range of meta-learning methods.
  arXiv  Detail & Related papers  (2022-10-11T06:45:15Z)
• Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks [62.63852372239708]
  Meta learning aims to learn the learning methodology for machine learning from observed tasks, so as to generalize to new query tasks. We interpret such learning methodology as learning an explicit hyper- parameter prediction function shared by all training tasks. Such setting guarantees that the meta-learned learning methodology is able to flexibly fit diverse query tasks.
  arXiv  Detail & Related papers  (2021-07-06T04:05:08Z)
• Trainable Class Prototypes for Few-Shot Learning [5.481942307939029]
  We propose the trainable prototypes for distance measure instead of the artificial ones within the meta-training and task-training framework. Also to avoid the disadvantages that the episodic meta-training brought, we adopt non-episodic meta-training based on self-supervised learning. Our method achieves state-of-the-art performance in a variety of established few-shot tasks on the standard few-shot visual classification dataset.
  arXiv  Detail & Related papers  (2021-06-21T04:19:56Z)
• Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? [72.00712736992618]
  We show that a simple baseline: learning a supervised or self-supervised representation on the meta-training set, outperforms state-of-the-art few-shot learning methods. An additional boost can be achieved through the use of self-distillation. We believe that our findings motivate a rethinking of few-shot image classification benchmarks and the associated role of meta-learning algorithms.
  arXiv  Detail & Related papers  (2020-03-25T17:58:42Z)
• Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning [79.25478727351604]
  We explore a simple process: meta-learning over a whole-classification pre-trained model on its evaluation metric. We observe this simple method achieves competitive performance to state-of-the-art methods on standard benchmarks.
  arXiv  Detail & Related papers  (2020-03-09T20:06:36Z)
• Towards explainable meta-learning [5.802346990263708]
  Meta-learning aims at discovering how different machine learning algorithms perform on a wide range of predictive tasks. State of the art approaches are focused on searching for the best meta-model but do not explain how these different aspects contribute to its performance. We propose techniques developed for eXplainable Artificial Intelligence (XAI) to examine and extract knowledge from black-box surrogate models.
  arXiv  Detail & Related papers  (2020-02-11T09:42:29Z)
• Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
  We develop a notion of incremental learning during the meta-training phase of meta-learning. Our approach performs favorably at test time as compared to training a model with the full meta-training set.
  arXiv  Detail & Related papers  (2020-02-11T01:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.