Connecting Context-specific Adaptation in Humans to Meta-learning
- URL: http://arxiv.org/abs/2011.13782v2
- Date: Tue, 1 Dec 2020 01:33:18 GMT
- Title: Connecting Context-specific Adaptation in Humans to Meta-learning
- Authors: Rachit Dubey, Erin Grant, Michael Luo, Karthik Narasimhan, Thomas
Griffiths
- Abstract summary: We show how context-conditioned meta-learning can capture human behavior in a cognitive task.
Our work demonstrates that guiding meta-learning with task information can capture complex, human-like behavior.
- Score: 23.923548278086383
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Cognitive control, the ability of a system to adapt to the demands of a task,
is an integral part of cognition. A widely accepted fact about cognitive
control is that it is context-sensitive: Adults and children alike infer
information about a task's demands from contextual cues and use these
inferences to learn from ambiguous cues. However, the precise way in which
people use contextual cues to guide adaptation to a new task remains poorly
understood. This work connects the context-sensitive nature of cognitive
control to a method for meta-learning with context-conditioned adaptation. We
begin by identifying an essential difference between human learning and current
approaches to meta-learning: In contrast to humans, existing meta-learning
algorithms do not make use of task-specific contextual cues but instead rely
exclusively on online feedback in the form of task-specific labels or rewards.
To remedy this, we introduce a framework for using contextual information about
a task to guide the initialization of task-specific models before adaptation to
online feedback. We show how context-conditioned meta-learning can capture
human behavior in a cognitive task and how it can be scaled to improve the
speed of learning in various settings, including few-shot classification and
low-sample reinforcement learning. Our work demonstrates that guiding
meta-learning with task information can capture complex, human-like behavior,
thereby deepening our understanding of cognitive control.
Related papers
- In-context Learning in Presence of Spurious Correlations [8.055478206164105]
We study the possibility of training an in-context learner for classification tasks involving spurious features.
We find that the conventional approach of training in-context learners is susceptible to spurious features.
We propose a novel technique to train such a learner for a given classification task.
arXiv Detail & Related papers (2024-10-04T04:26:36Z) - Exploring CausalWorld: Enhancing robotic manipulation via knowledge transfer and curriculum learning [6.683222869973898]
This study explores a learning-based tri-finger robotic arm manipulating task, which requires complex movements and coordination among the fingers.
By employing reinforcement learning, we train an agent to acquire the necessary skills for proficient manipulation.
Two knowledge transfer strategies, fine-tuning and curriculum learning, were utilized within the soft actor-critic architecture.
arXiv Detail & Related papers (2024-03-25T23:19:19Z) - Choreographer: Learning and Adapting Skills in Imagination [60.09911483010824]
We present Choreographer, a model-based agent that exploits its world model to learn and adapt skills in imagination.
Our method decouples the exploration and skill learning processes, being able to discover skills in the latent state space of the model.
Choreographer is able to learn skills both from offline data, and by collecting data simultaneously with an exploration policy.
arXiv Detail & Related papers (2022-11-23T23:31:14Z) - Rethinking Learning Dynamics in RL using Adversarial Networks [79.56118674435844]
We present a learning mechanism for reinforcement learning of closely related skills parameterized via a skill embedding space.
The main contribution of our work is to formulate an adversarial training regime for reinforcement learning with the help of entropy-regularized policy gradient formulation.
arXiv Detail & Related papers (2022-01-27T19:51:09Z) - Meta-learning the Learning Trends Shared Across Tasks [123.10294801296926]
Gradient-based meta-learning algorithms excel at quick adaptation to new tasks with limited data.
Existing meta-learning approaches only depend on the current task information during the adaptation.
We propose a 'Path-aware' model-agnostic meta-learning approach.
arXiv Detail & Related papers (2020-10-19T08:06:47Z) - Importance Weighted Policy Learning and Adaptation [89.46467771037054]
We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning.
The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior.
Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
arXiv Detail & Related papers (2020-09-10T14:16:58Z) - Concept Learners for Few-Shot Learning [76.08585517480807]
We propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions.
We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation.
arXiv Detail & Related papers (2020-07-14T22:04:17Z) - Transforming task representations to perform novel tasks [12.008469282323492]
An important aspect of intelligence is the ability to adapt to a novel task without any direct experience (zero-shot)
We propose a general computational framework for adapting to novel tasks based on their relationship to prior tasks.
arXiv Detail & Related papers (2020-05-08T23:41:57Z) - Automated Relational Meta-learning [95.02216511235191]
We propose an automated relational meta-learning framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph.
We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.
arXiv Detail & Related papers (2020-01-03T07:02:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.