Uncertainty-Aware Meta-Learning for Multimodal Task Distributions
- URL: http://arxiv.org/abs/2210.01881v1
- Date: Tue, 4 Oct 2022 20:02:25 GMT
- Title: Uncertainty-Aware Meta-Learning for Multimodal Task Distributions
- Authors: Cesar Almecija, Apoorva Sharma and Navid Azizan
- Abstract summary: We present UnLiMiTD (uncertainty-aware meta-learning for multimodal task distributions)
We take a probabilistic perspective and train a parametric, tuneable distribution over tasks on the meta-dataset.
We demonstrate that UnLiMiTD's predictions compare favorably to, and outperform in most cases, the standard baselines.
- Score: 3.7470451129384825
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Meta-learning or learning to learn is a popular approach for learning new
tasks with limited data (i.e., few-shot learning) by leveraging the
commonalities among different tasks. However, meta-learned models can perform
poorly when context data is limited, or when data is drawn from an
out-of-distribution (OoD) task. Especially in safety-critical settings, this
necessitates an uncertainty-aware approach to meta-learning. In addition, the
often multimodal nature of task distributions can pose unique challenges to
meta-learning methods. In this work, we present UnLiMiTD (uncertainty-aware
meta-learning for multimodal task distributions), a novel method for
meta-learning that (1) makes probabilistic predictions on in-distribution tasks
efficiently, (2) is capable of detecting OoD context data at test time, and (3)
performs on heterogeneous, multimodal task distributions. To achieve this goal,
we take a probabilistic perspective and train a parametric, tuneable
distribution over tasks on the meta-dataset. We construct this distribution by
performing Bayesian inference on a linearized neural network, leveraging
Gaussian process theory. We demonstrate that UnLiMiTD's predictions compare
favorably to, and outperform in most cases, the standard baselines, especially
in the low-data regime. Furthermore, we show that UnLiMiTD is effective in
detecting data from OoD tasks. Finally, we confirm that both of these findings
continue to hold in the multimodal task-distribution setting.
Related papers
- Exploring intra-task relations to improve meta-learning algorithms [1.223779595809275]
We aim to exploit external knowledge of task relations to improve training stability via effective mini-batching of tasks.
We hypothesize that selecting a diverse set of tasks in a mini-batch will lead to a better estimate of the full gradient and hence will lead to a reduction of noise in training.
arXiv Detail & Related papers (2023-12-27T15:33:52Z) - Task-Distributionally Robust Data-Free Meta-Learning [99.56612787882334]
Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data.
For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift ( TDS) and Task-Distribution Corruption (TDC)
arXiv Detail & Related papers (2023-11-23T15:46:54Z) - Diffusion Model is an Effective Planner and Data Synthesizer for
Multi-Task Reinforcement Learning [101.66860222415512]
Multi-Task Diffusion Model (textscMTDiff) is a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis.
For generative planning, we find textscMTDiff outperforms state-of-the-art algorithms across 50 tasks on Meta-World and 8 maps on Maze2D.
arXiv Detail & Related papers (2023-05-29T05:20:38Z) - Algorithm Design for Online Meta-Learning with Task Boundary Detection [63.284263611646]
We propose a novel algorithm for task-agnostic online meta-learning in non-stationary environments.
We first propose two simple but effective detection mechanisms of task switches and distribution shift.
We show that a sublinear task-averaged regret can be achieved for our algorithm under mild conditions.
arXiv Detail & Related papers (2023-02-02T04:02:49Z) - The Effect of Diversity in Meta-Learning [79.56118674435844]
Few-shot learning aims to learn representations that can tackle novel tasks given a small number of examples.
Recent studies show that task distribution plays a vital role in the model's performance.
We study different task distributions on a myriad of models and datasets to evaluate the effect of task diversity on meta-learning algorithms.
arXiv Detail & Related papers (2022-01-27T19:39:07Z) - Generating meta-learning tasks to evolve parametric loss for
classification learning [1.1355370218310157]
In existing meta-learning approaches, learning tasks for training meta-models are usually collected from public datasets.
We propose a meta-learning approach based on randomly generated meta-learning tasks to obtain a parametric loss for classification learning based on big data.
arXiv Detail & Related papers (2021-11-20T13:07:55Z) - Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP [39.457091182683406]
We aim to provide task distributions for meta-learning by considering self-supervised tasks automatically proposed from unlabeled text.
Our analysis shows that all these factors meaningfully alter the task distribution, some inducing significant improvements in downstream few-shot accuracy of the meta-learned models.
arXiv Detail & Related papers (2021-11-02T01:50:09Z) - A Channel Coding Benchmark for Meta-Learning [21.2424398453955]
Several important issues in meta-learning have proven hard to study thus far.
We propose the channel coding problem as a benchmark for meta-learning.
Going forward, this benchmark provides a tool for the community to study the capabilities and limitations of meta-learning.
arXiv Detail & Related papers (2021-07-15T19:37:43Z) - Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML.
We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML.
Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
arXiv Detail & Related papers (2020-10-06T16:51:09Z) - Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction.
We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data.
Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
arXiv Detail & Related papers (2020-03-10T03:10:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.