Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification

• URL: http://arxiv.org/abs/2007.06240v1
• Date: Mon, 13 Jul 2020 08:49:00 GMT
• Title: Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• Authors: Yucan Zhou, Yu Wang, Jianfei Cai, Yu Zhou, Qinghua Hu, Weiping Wang
• Abstract summary: We propose an easy-to-hard expert meta-training strategy to arrange the training tasks properly. A task hardness aware module is designed and integrated into the training procedure to estimate the hardness of a task. Experimental results on the miniImageNet and tieredImageNetSketch datasets show that the meta-learners can obtain better results with our expert training strategy.
• Score: 62.10696018098057
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks are highly effective when a large number of labeled samples are available but fail with few-shot classification tasks. Recently, meta-learning methods have received much attention, which train a meta-learner on massive additional tasks to gain the knowledge to instruct the few-shot classification. Usually, the training tasks are randomly sampled and performed indiscriminately, often making the meta-learner stuck into a bad local optimum. Some works in the optimization of deep neural networks have shown that a better arrangement of training data can make the classifier converge faster and perform better. Inspired by this idea, we propose an easy-to-hard expert meta-training strategy to arrange the training tasks properly, where easy tasks are preferred in the first phase, then, hard tasks are emphasized in the second phase. A task hardness aware module is designed and integrated into the training procedure to estimate the hardness of a task based on the distinguishability of its categories. In addition, we explore multiple hardness measurements including the semantic relation, the pairwise Euclidean distance, the Hausdorff distance, and the Hilbert-Schmidt independence criterion. Experimental results on the miniImageNet and tieredImageNetSketch datasets show that the meta-learners can obtain better results with our expert training strategy.

Related papers

• CPT: Competence-progressive Training Strategy for Few-shot Node Classification [11.17199104891692]
  Graph Neural Networks (GNNs) have made significant advancements in node classification, but their success relies on sufficient labeled nodes per class in the training data. Traditional episodic meta-learning approaches have shown promise in this domain, but they face an inherent limitation. We introduce CPT, a novel two-stage curriculum learning method that aligns task difficulty with the meta-learner's progressive competence.
  arXiv  Detail & Related papers  (2024-02-01T09:36:56Z)
• Episodic-free Task Selection for Few-shot Learning [2.508902852545462]
  We propose a novel meta-training framework beyond episodic training. episodic tasks are not used directly for training, but for evaluating the effectiveness of some selected episodic-free tasks. In experiments, the training task set contains some promising types, e. g., contrastive learning and classification.
  arXiv  Detail & Related papers  (2024-01-31T10:52:15Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning [71.55412580325743]
  We show that multi-task pretraining with fine-tuning on new tasks performs equally as well, or better, than meta-pretraining with meta test-time adaptation. This is encouraging for future research, as multi-task pretraining tends to be simpler and computationally cheaper than meta-RL.
  arXiv  Detail & Related papers  (2022-06-07T13:24:00Z)
• Generating meta-learning tasks to evolve parametric loss for classification learning [1.1355370218310157]
  In existing meta-learning approaches, learning tasks for training meta-models are usually collected from public datasets. We propose a meta-learning approach based on randomly generated meta-learning tasks to obtain a parametric loss for classification learning based on big data.
  arXiv  Detail & Related papers  (2021-11-20T13:07:55Z)
• MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning [46.79356071007187]
  We propose a holistic approach to jointly train the backbone network and the channel gating. We develop a federated meta-learning approach to jointly learn good meta-initializations for both backbone networks and gating modules.
  arXiv  Detail & Related papers  (2020-11-25T04:26:23Z)
• Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
  We introduce a fast optimization-based meta-learning method for few-shot classification. Our strategy enables important aspects of the base learner objective to be learned during meta-training. We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-01T15:59:31Z)
• Adaptive Task Sampling for Meta-Learning [79.61146834134459]
  Key idea of meta-learning for few-shot classification is to mimic the few-shot situations faced at test time. We propose an adaptive task sampling method to improve the generalization performance.
  arXiv  Detail & Related papers  (2020-07-17T03:15:53Z)
• Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
  We develop a notion of incremental learning during the meta-training phase of meta-learning. Our approach performs favorably at test time as compared to training a model with the full meta-training set.
  arXiv  Detail & Related papers  (2020-02-11T01:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.