Meta-Learning across Meta-Tasks for Few-Shot Learning

• URL: http://arxiv.org/abs/2002.04274v4
• Date: Sat, 26 Sep 2020 05:02:10 GMT
• Title: Meta-Learning across Meta-Tasks for Few-Shot Learning
• Authors: Nanyi Fei, Zhiwu Lu, Yizhao Gao, Jia Tian, Tao Xiang and Ji-Rong Wen
• Abstract summary: We argue that the inter-meta-task relationships should be exploited and those tasks are sampled strategically to assist in meta-learning. We consider the relationships defined over two types of meta-task pairs and propose different strategies to exploit them.
• Score: 107.44950540552765
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing meta-learning based few-shot learning (FSL) methods typically adopt an episodic training strategy whereby each episode contains a meta-task. Across episodes, these tasks are sampled randomly and their relationships are ignored. In this paper, we argue that the inter-meta-task relationships should be exploited and those tasks are sampled strategically to assist in meta-learning. Specifically, we consider the relationships defined over two types of meta-task pairs and propose different strategies to exploit them. (1) Two meta-tasks with disjoint sets of classes: this pair is interesting because it is reminiscent of the relationship between the source seen classes and target unseen classes, featured with domain gap caused by class differences. A novel learning objective termed meta-domain adaptation (MDA) is proposed to make the meta-learned model more robust to the domain gap. (2) Two meta-tasks with identical sets of classes: this pair is useful because it can be employed to learn models that are robust against poorly sampled few-shots. To that end, a novel meta-knowledge distillation (MKD) objective is formulated. There are some mistakes in the experiments. We thus choose to withdraw this paper.

Related papers

• MetaICL: Learning to Learn In Context [87.23056864536613]
  We introduce MetaICL, a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learn-ing on a large set of training tasks. We show that MetaICL approaches (and sometimes beats) the performance of models fully finetuned on the target task training data, and outperforms much bigger models with nearly 8x parameters.
  arXiv  Detail & Related papers  (2021-10-29T17:42:08Z)
• Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data [95.47859525676246]
  A recent study finds that existing few-shot learning methods, trained on the source domain, fail to generalize to the novel target domain when a domain gap is observed. In this paper, we realize that the labeled target data in Cross-Domain Few-Shot Learning has not been leveraged in any way to help the learning process.
  arXiv  Detail & Related papers  (2021-07-26T06:15:45Z)
• Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling [29.218780709974244]
  We propose a few-shot learning scheme that learns to bridge metric spaces of intent and slot on data-rich domains. Our model significantly outperforms the strong baselines in one and five shots settings.
  arXiv  Detail & Related papers  (2021-05-25T15:07:11Z)
• Meta-Regularization by Enforcing Mutual-Exclusiveness [0.8057006406834467]
  We propose a regularization technique for meta-learning models that gives the model designer more control over the information flow during meta-training. Our proposed regularization function shows an accuracy boost of $sim$ $36%$ on the Omniglot dataset.
  arXiv  Detail & Related papers  (2021-01-24T22:57:19Z)
• Yet Meta Learning Can Adapt Fast, It Can Also Break Easily [53.65787902272109]
  We study adversarial attacks on meta learning under the few-shot classification problem. We propose the first attacking algorithm against meta learning under various settings.
  arXiv  Detail & Related papers  (2020-09-02T15:03:14Z)
• Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation [26.296412053816233]
  We propose a meta-learning framework for few-shot word sense disambiguation. The goal is to learn to disambiguate unseen words from only a few labeled instances. We extend several popular meta-learning approaches to this scenario, and analyze their strengths and weaknesses.
  arXiv  Detail & Related papers  (2020-04-29T17:33:31Z)
• Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning [79.25478727351604]
  We explore a simple process: meta-learning over a whole-classification pre-trained model on its evaluation metric. We observe this simple method achieves competitive performance to state-of-the-art methods on standard benchmarks.
  arXiv  Detail & Related papers  (2020-03-09T20:06:36Z)
• Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
  We develop a notion of incremental learning during the meta-training phase of meta-learning. Our approach performs favorably at test time as compared to training a model with the full meta-training set.
  arXiv  Detail & Related papers  (2020-02-11T01:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.