Bayesian Meta-Learning Through Variational Gaussian Processes
- URL: http://arxiv.org/abs/2110.11044v1
- Date: Thu, 21 Oct 2021 10:44:23 GMT
- Title: Bayesian Meta-Learning Through Variational Gaussian Processes
- Authors: Vivek Myers, Nikhil Sardana
- Abstract summary: We extend Gaussian-process-based meta-learning to allow for high-quality, arbitrary non-Gaussian uncertainty predictions.
Our method performs significantly better than existing Bayesian meta-learning baselines.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advances in the field of meta-learning have tackled domains consisting
of large numbers of small ("few-shot") supervised learning tasks. Meta-learning
algorithms must be able to rapidly adapt to any individual few-shot task,
fitting to a small support set within a task and using it to predict the labels
of the task's query set. This problem setting can be extended to the Bayesian
context, wherein rather than predicting a single label for each query data
point, a model predicts a distribution of labels capturing its uncertainty.
Successful methods in this domain include Bayesian ensembling of MAML-based
models, Bayesian neural networks, and Gaussian processes with learned deep
kernel and mean functions. While Gaussian processes have a robust Bayesian
interpretation in the meta-learning context, they do not naturally model
non-Gaussian predictive posteriors for expressing uncertainty. In this paper,
we design a theoretically principled method, VMGP, extending
Gaussian-process-based meta-learning to allow for high-quality, arbitrary
non-Gaussian uncertainty predictions. On benchmark environments with complex
non-smooth or discontinuous structure, we find our VMGP method performs
significantly better than existing Bayesian meta-learning baselines.
Related papers
- Meta-learning to Calibrate Gaussian Processes with Deep Kernels for
Regression Uncertainty Estimation [43.23399636191726]
We propose a meta-learning method for calibrating deep kernel GPs for improving regression uncertainty estimation performance.
The proposed method meta-learns how to calibrate uncertainty using data from various tasks by minimizing the test expected calibration error.
Our experiments demonstrate that the proposed method improves uncertainty estimation performance while keeping high regression performance.
arXiv Detail & Related papers (2023-12-13T07:58:47Z) - Scalable Meta-Learning with Gaussian Processes [11.528128570533273]
We develop ScaML-GP, a modular GP model for meta-learning that is scalable in the number of tasks.
Our core contribution is a carefully designed multi-task kernel that enables hierarchical training and task scalability.
In synthetic and real-world meta-learning experiments, we demonstrate that ScaML-GP can learn efficiently both with few and many meta-tasks.
arXiv Detail & Related papers (2023-12-01T17:25:10Z) - Scalable Bayesian Meta-Learning through Generalized Implicit Gradients [64.21628447579772]
Implicit Bayesian meta-learning (iBaML) method broadens the scope of learnable priors, but also quantifies the associated uncertainty.
Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one.
arXiv Detail & Related papers (2023-03-31T02:10:30Z) - MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
We present a novel approach to extracting inductive biases from a set of related datasets.
We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space.
Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
arXiv Detail & Related papers (2022-10-24T15:14:26Z) - Incremental Ensemble Gaussian Processes [53.3291389385672]
We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary.
With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions.
The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
arXiv Detail & Related papers (2021-10-13T15:11:25Z) - Transfer Bayesian Meta-learning via Weighted Free Energy Minimization [37.51664463278401]
A key assumption is that the auxiliary tasks, known as meta-training tasks, share the same generating distribution as the tasks to be encountered at deployment time.
This paper introduces weighted free energy minimization (WFEM) for transfer meta-learning.
arXiv Detail & Related papers (2021-06-20T15:17:51Z) - Covariate Distribution Aware Meta-learning [3.494950334697974]
We propose a computationally feasible meta-learning algorithm by introducing meaningful relaxations.
We demonstrate the gains of our algorithm over bootstrapped based meta-learning baselines on popular classification benchmarks.
arXiv Detail & Related papers (2020-07-06T05:00:13Z) - Meta Cyclical Annealing Schedule: A Simple Approach to Avoiding
Meta-Amortization Error [50.83356836818667]
We develop a novel meta-regularization objective using it cyclical annealing schedule and it maximum mean discrepancy (MMD) criterion.
The experimental results show that our approach substantially outperforms standard meta-learning algorithms.
arXiv Detail & Related papers (2020-03-04T04:43:16Z) - Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples.
We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries.
We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
arXiv Detail & Related papers (2020-02-27T10:22:17Z) - PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees [77.67258935234403]
We provide a theoretical analysis using the PAC-Bayesian framework and derive novel generalization bounds for meta-learning.
We develop a class of PAC-optimal meta-learning algorithms with performance guarantees and a principled meta-level regularization.
arXiv Detail & Related papers (2020-02-13T15:01:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.