Improving Generalization in Meta-learning via Task Augmentation
- URL: http://arxiv.org/abs/2007.13040v3
- Date: Wed, 9 Jun 2021 19:19:01 GMT
- Title: Improving Generalization in Meta-learning via Task Augmentation
- Authors: Huaxiu Yao, Longkai Huang, Linjun Zhang, Ying Wei, Li Tian, James Zou,
Junzhou Huang, Zhenhui Li
- Abstract summary: We propose two task augmentation methods, including MetaMix and Channel Shuffle.
Both MetaMix and Channel Shuffle outperform state-of-the-art results by a large margin across many datasets.
- Score: 69.83677015207527
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Meta-learning has proven to be a powerful paradigm for transferring the
knowledge from previous tasks to facilitate the learning of a novel task.
Current dominant algorithms train a well-generalized model initialization which
is adapted to each task via the support set. The crux lies in optimizing the
generalization capability of the initialization, which is measured by the
performance of the adapted model on the query set of each task. Unfortunately,
this generalization measure, evidenced by empirical results, pushes the
initialization to overfit the meta-training tasks, which significantly impairs
the generalization and adaptation to novel tasks. To address this issue, we
actively augment a meta-training task with "more data" when evaluating the
generalization. Concretely, we propose two task augmentation methods, including
MetaMix and Channel Shuffle. MetaMix linearly combines features and labels of
samples from both the support and query sets. For each class of samples,
Channel Shuffle randomly replaces a subset of their channels with the
corresponding ones from a different class. Theoretical studies show how task
augmentation improves the generalization of meta-learning. Moreover, both
MetaMix and Channel Shuffle outperform state-of-the-art results by a large
margin across many datasets and are compatible with existing meta-learning
algorithms.
Related papers
- Meta-Learning with Heterogeneous Tasks [42.695853959923625]
Heterogeneous Tasks Robust Meta-learning (HeTRoM)
An efficient iterative optimization algorithm based on bi-level optimization.
Results demonstrate that our method provides flexibility, enabling users to adapt to diverse task settings.
arXiv Detail & Related papers (2024-10-24T16:32:23Z) - Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios.
By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels.
Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
arXiv Detail & Related papers (2021-06-04T20:15:34Z) - Large-Scale Meta-Learning with Continual Trajectory Shifting [76.29017270864308]
We show that allowing the meta-learners to take a larger number of inner gradient steps better captures the structure of heterogeneous and large-scale tasks.
In order to increase the frequency of meta-updates, we propose to estimate the required shift of the task-specific parameters.
We show that the algorithm largely outperforms the previous first-order meta-learning methods in terms of both generalization performance and convergence.
arXiv Detail & Related papers (2021-02-14T18:36:33Z) - Meta-Regularization by Enforcing Mutual-Exclusiveness [0.8057006406834467]
We propose a regularization technique for meta-learning models that gives the model designer more control over the information flow during meta-training.
Our proposed regularization function shows an accuracy boost of $sim$ $36%$ on the Omniglot dataset.
arXiv Detail & Related papers (2021-01-24T22:57:19Z) - Variable-Shot Adaptation for Online Meta-Learning [123.47725004094472]
We study the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.
We find that meta-learning solves the full task set with fewer overall labels and greater cumulative performance, compared to standard supervised methods.
These results suggest that meta-learning is an important ingredient for building learning systems that continuously learn and improve over a sequence of problems.
arXiv Detail & Related papers (2020-12-14T18:05:24Z) - Adaptive Task Sampling for Meta-Learning [79.61146834134459]
Key idea of meta-learning for few-shot classification is to mimic the few-shot situations faced at test time.
We propose an adaptive task sampling method to improve the generalization performance.
arXiv Detail & Related papers (2020-07-17T03:15:53Z) - Information-Theoretic Generalization Bounds for Meta-Learning and
Applications [42.275148861039895]
Key performance measure for meta-learning is the meta-generalization gap.
This paper presents novel information-theoretic upper bounds on the meta-generalization gap.
arXiv Detail & Related papers (2020-05-09T05:48:01Z) - Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
We develop a notion of incremental learning during the meta-training phase of meta-learning.
Our approach performs favorably at test time as compared to training a model with the full meta-training set.
arXiv Detail & Related papers (2020-02-11T01:39:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.