Meta-Learning of Structured Task Distributions in Humans and Machines
- URL: http://arxiv.org/abs/2010.02317v3
- Date: Thu, 18 Mar 2021 13:42:14 GMT
- Title: Meta-Learning of Structured Task Distributions in Humans and Machines
- Authors: Sreejan Kumar, Ishita Dasgupta, Jonathan D. Cohen, Nathaniel D. Daw,
Thomas L. Griffiths
- Abstract summary: We show that evaluating meta-learning remains a challenge, and can miss whether meta-learning actually uses the structure embedded within the tasks.
We train a standard meta-learning agent, a recurrent network trained with model-free reinforcement learning, and compare it with human performance.
We find a double dissociation in which humans do better in the structured task distribution whereas agents do better in the null task distribution.
- Score: 15.34209852089588
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, meta-learning, in which a model is trained on a family of
tasks (i.e. a task distribution), has emerged as an approach to training neural
networks to perform tasks that were previously assumed to require structured
representations, making strides toward closing the gap between humans and
machines. However, we argue that evaluating meta-learning remains a challenge,
and can miss whether meta-learning actually uses the structure embedded within
the tasks. These meta-learners might therefore still be significantly different
from humans learners. To demonstrate this difference, we first define a new
meta-reinforcement learning task in which a structured task distribution is
generated using a compositional grammar. We then introduce a novel approach to
constructing a "null task distribution" with the same statistical complexity as
this structured task distribution but without the explicit rule-based structure
used to generate the structured task. We train a standard meta-learning agent,
a recurrent network trained with model-free reinforcement learning, and compare
it with human performance across the two task distributions. We find a double
dissociation in which humans do better in the structured task distribution
whereas agents do better in the null task distribution -- despite comparable
statistical complexity. This work highlights that multiple strategies can
achieve reasonable meta-test performance, and that careful construction of
control task distributions is a valuable way to understand which strategies
meta-learners acquire, and how they might differ from humans.
Related papers
- Meta-Learning with Heterogeneous Tasks [42.695853959923625]
Heterogeneous Tasks Robust Meta-learning (HeTRoM)
An efficient iterative optimization algorithm based on bi-level optimization.
Results demonstrate that our method provides flexibility, enabling users to adapt to diverse task settings.
arXiv Detail & Related papers (2024-10-24T16:32:23Z) - ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning [49.447777286862994]
ConML is a universal meta-learning framework that can be applied to various meta-learning algorithms.
We demonstrate that ConML integrates seamlessly with optimization-based, metric-based, and amortization-based meta-learning algorithms.
arXiv Detail & Related papers (2024-10-08T12:22:10Z) - Task-Distributionally Robust Data-Free Meta-Learning [99.56612787882334]
Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data.
For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift ( TDS) and Task-Distribution Corruption (TDC)
arXiv Detail & Related papers (2023-11-23T15:46:54Z) - Fast Inference and Transfer of Compositional Task Structures for
Few-shot Task Generalization [101.72755769194677]
We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph.
Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks.
Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
arXiv Detail & Related papers (2022-05-25T10:44:25Z) - MetaICL: Learning to Learn In Context [87.23056864536613]
We introduce MetaICL, a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learn-ing on a large set of training tasks.
We show that MetaICL approaches (and sometimes beats) the performance of models fully finetuned on the target task training data, and outperforms much bigger models with nearly 8x parameters.
arXiv Detail & Related papers (2021-10-29T17:42:08Z) - Latent Group Structured Multi-task Learning [2.827177139912107]
In multi-task learning (MTL), we improve the performance of key machine learning algorithms by training various tasks jointly.
We present our group structured latent-space multi-task learning model, which encourages group structured tasks defined by prior information.
Experiments are conducted on both synthetic and real-world datasets, showing competitive performance over single-task learning.
arXiv Detail & Related papers (2020-11-24T05:38:58Z) - Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
We develop a notion of incremental learning during the meta-training phase of meta-learning.
Our approach performs favorably at test time as compared to training a model with the full meta-training set.
arXiv Detail & Related papers (2020-02-11T01:39:12Z) - Automated Relational Meta-learning [95.02216511235191]
We propose an automated relational meta-learning framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph.
We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.
arXiv Detail & Related papers (2020-01-03T07:02:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.