MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Tasks

• URL: http://arxiv.org/abs/2305.10309v1
• Date: Wed, 17 May 2023 15:47:47 GMT
• Title: MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Tasks
• Authors: Wenfang Sun, Yingjun Du, Xiantong Zhen, Fan Wang, Ling Wang, Cees G.M. Snoek
• Abstract summary: We propose a method for few-shot learning with fewer tasks, which is by metaulation. We modify parameters at various batch levels to increase the meta-training tasks. We also introduce learning variational feature hierarchies by incorporating the variationalulation.
• Score: 63.016244188951696
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-learning algorithms are able to learn a new task using previously learned knowledge, but they often require a large number of meta-training tasks which may not be readily available. To address this issue, we propose a method for few-shot learning with fewer tasks, which we call MetaModulation. The key idea is to use a neural network to increase the density of the meta-training tasks by modulating batch normalization parameters during meta-training. Additionally, we modify parameters at various network levels, rather than just a single layer, to increase task diversity. To account for the uncertainty caused by the limited training tasks, we propose a variational MetaModulation where the modulation parameters are treated as latent variables. We also introduce learning variational feature hierarchies by the variational MetaModulation, which modulates features at all layers and can consider task uncertainty and generate more diverse tasks. The ablation studies illustrate the advantages of utilizing a learnable task modulation at different levels and demonstrate the benefit of incorporating probabilistic variants in few-task meta-learning. Our MetaModulation and its variational variants consistently outperform state-of-the-art alternatives on four few-task meta-learning benchmarks.

Related papers

• Meta-Learning with Heterogeneous Tasks [42.695853959923625]
  Heterogeneous Tasks Robust Meta-learning (HeTRoM) An efficient iterative optimization algorithm based on bi-level optimization. Results demonstrate that our method provides flexibility, enabling users to adapt to diverse task settings.
  arXiv  Detail & Related papers  (2024-10-24T16:32:23Z)
• Towards Task Sampler Learning for Meta-Learning [37.02030832662183]
  Meta-learning aims to learn general knowledge with diverse training tasks conducted from limited data, and then transfer it to new tasks. It is commonly believed that increasing task diversity will enhance the generalization ability of meta-learning models. This paper challenges this view through empirical and theoretical analysis.
  arXiv  Detail & Related papers  (2023-07-18T01:53:18Z)
• Invariant Meta Learning for Out-of-Distribution Generalization [1.1718589131017048]
  In this paper, we propose invariant meta learning for out-of-distribution tasks. Specifically, invariant optimal meta-initialization,and fast adapt to out-of-distribution tasks with regularization penalty.
  arXiv  Detail & Related papers  (2023-01-26T12:53:21Z)
• Contrastive Knowledge-Augmented Meta-Learning for Few-Shot Classification [28.38744876121834]
  We introduce CAML (Contrastive Knowledge-Augmented Meta Learning), a novel approach for knowledge-enhanced few-shot learning. We evaluate the performance of CAML in different few-shot learning scenarios.
  arXiv  Detail & Related papers  (2022-07-25T17:01:29Z)
• Set-based Meta-Interpolation for Few-Task Meta-Learning [79.4236527774689]
  We propose a novel domain-agnostic task augmentation method, Meta-Interpolation, to densify the meta-training task distribution. We empirically validate the efficacy of Meta-Interpolation on eight datasets spanning across various domains.
  arXiv  Detail & Related papers  (2022-05-20T06:53:03Z)
• Fully Online Meta-Learning Without Task Boundaries [80.09124768759564]
  We study how meta-learning can be applied to tackle online problems of this nature. We propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries. Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods.
  arXiv  Detail & Related papers  (2022-02-01T07:51:24Z)
• The Effect of Diversity in Meta-Learning [79.56118674435844]
  Few-shot learning aims to learn representations that can tackle novel tasks given a small number of examples. Recent studies show that task distribution plays a vital role in the model's performance. We study different task distributions on a myriad of models and datasets to evaluate the effect of task diversity on meta-learning algorithms.
  arXiv  Detail & Related papers  (2022-01-27T19:39:07Z)
• MetaICL: Learning to Learn In Context [87.23056864536613]
  We introduce MetaICL, a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learn-ing on a large set of training tasks. We show that MetaICL approaches (and sometimes beats) the performance of models fully finetuned on the target task training data, and outperforms much bigger models with nearly 8x parameters.
  arXiv  Detail & Related papers  (2021-10-29T17:42:08Z)
• A Channel Coding Benchmark for Meta-Learning [21.2424398453955]
  Several important issues in meta-learning have proven hard to study thus far. We propose the channel coding problem as a benchmark for meta-learning. Going forward, this benchmark provides a tool for the community to study the capabilities and limitations of meta-learning.
  arXiv  Detail & Related papers  (2021-07-15T19:37:43Z)
• Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
  Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios. By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels. Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
  arXiv  Detail & Related papers  (2021-06-04T20:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.