Transfer Bayesian Meta-learning via Weighted Free Energy Minimization

• URL: http://arxiv.org/abs/2106.10711v2
• Date: Tue, 22 Jun 2021 08:48:51 GMT
• Title: Transfer Bayesian Meta-learning via Weighted Free Energy Minimization
• Authors: Yunchuan Zhang, Sharu Theresa Jose, Osvaldo Simeone
• Abstract summary: A key assumption is that the auxiliary tasks, known as meta-training tasks, share the same generating distribution as the tasks to be encountered at deployment time. This paper introduces weighted free energy minimization (WFEM) for transfer meta-learning.
• Score: 37.51664463278401
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Meta-learning optimizes the hyperparameters of a training procedure, such as its initialization, kernel, or learning rate, based on data sampled from a number of auxiliary tasks. A key underlying assumption is that the auxiliary tasks, known as meta-training tasks, share the same generating distribution as the tasks to be encountered at deployment time, known as meta-test tasks. This may, however, not be the case when the test environment differ from the meta-training conditions. To address shifts in task generating distribution between meta-training and meta-testing phases, this paper introduces weighted free energy minimization (WFEM) for transfer meta-learning. We instantiate the proposed approach for non-parametric Bayesian regression and classification via Gaussian Processes (GPs). The method is validated on a toy sinusoidal regression problem, as well as on classification using miniImagenet and CUB data sets, through comparison with standard meta-learning of GP priors as implemented by PACOH.

Related papers

• Scalable Meta-Learning with Gaussian Processes [11.528128570533273]
  We develop ScaML-GP, a modular GP model for meta-learning that is scalable in the number of tasks. Our core contribution is a carefully designed multi-task kernel that enables hierarchical training and task scalability. In synthetic and real-world meta-learning experiments, we demonstrate that ScaML-GP can learn efficiently both with few and many meta-tasks.
  arXiv  Detail & Related papers  (2023-12-01T17:25:10Z)
• Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning [119.70303730341938]
  We propose ePisode cUrriculum inveRsion (ECI) during data-free meta training and invErsion calibRation following inner loop (ICFIL) during meta testing. ECI adaptively increases the difficulty level of pseudo episodes according to the real-time feedback of the meta model. We formulate the optimization process of meta training with ECI as an adversarial form in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-03-20T15:10:41Z)
• MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
  We present a novel approach to extracting inductive biases from a set of related datasets. We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space. Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
  arXiv  Detail & Related papers  (2022-10-24T15:14:26Z)
• Bayesian Meta-Learning Through Variational Gaussian Processes [0.0]
  We extend Gaussian-process-based meta-learning to allow for high-quality, arbitrary non-Gaussian uncertainty predictions. Our method performs significantly better than existing Bayesian meta-learning baselines.
  arXiv  Detail & Related papers  (2021-10-21T10:44:23Z)
• BAMLD: Bayesian Active Meta-Learning by Disagreement [39.59987601426039]
  This paper introduces an information-theoretic active task selection mechanism to decrease the number of labeling requests for meta-training tasks. We report its empirical performance results that compare favourably against existing acquisition mechanisms.
  arXiv  Detail & Related papers  (2021-10-19T13:06:51Z)
• Energy-Efficient and Federated Meta-Learning via Projected Stochastic Gradient Ascent [79.58680275615752]
  We propose an energy-efficient federated meta-learning framework. We assume each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model.
  arXiv  Detail & Related papers  (2021-05-31T08:15:44Z)
• An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning [44.320945743871285]
  We present novel information-theoretic bounds on the average absolute value of the meta-generalization gap. Our bounds explicitly capture the impact of task relatedness, the number of tasks, and the number of data samples per task on the meta-generalization gap.
  arXiv  Detail & Related papers  (2021-01-21T01:38:16Z)
• Transfer Meta-Learning: Information-Theoretic Bounds and Information Meta-Risk Minimization [47.7605527786164]
  Meta-learning automatically infers an inductive bias by observing data from a number of related tasks. We introduce the problem of transfer meta-learning, in which tasks are drawn from a target task environment during meta-testing.
  arXiv  Detail & Related papers  (2020-11-04T12:55:43Z)
• Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
  We develop a notion of incremental learning during the meta-training phase of meta-learning. Our approach performs favorably at test time as compared to training a model with the full meta-training set.
  arXiv  Detail & Related papers  (2020-02-11T01:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.