Fully Online Meta-Learning Without Task Boundaries
- URL: http://arxiv.org/abs/2202.00263v1
- Date: Tue, 1 Feb 2022 07:51:24 GMT
- Title: Fully Online Meta-Learning Without Task Boundaries
- Authors: Jathushan Rajasegaran, Chesea Finn, Sergey Levine
- Abstract summary: We study how meta-learning can be applied to tackle online problems of this nature.
We propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries.
Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods.
- Score: 80.09124768759564
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While deep networks can learn complex functions such as classifiers,
detectors, and trackers, many applications require models that continually
adapt to changing input distributions, changing tasks, and changing
environmental conditions. Indeed, this ability to continuously accrue knowledge
and use past experience to learn new tasks quickly in continual settings is one
of the key properties of an intelligent system. For complex and
high-dimensional problems, simply updating the model continually with standard
learning algorithms such as gradient descent may result in slow adaptation.
Meta-learning can provide a powerful tool to accelerate adaptation yet is
conventionally studied in batch settings. In this paper, we study how
meta-learning can be applied to tackle online problems of this nature,
simultaneously adapting to changing tasks and input distributions and
meta-training the model in order to adapt more quickly in the future. Extending
meta-learning into the online setting presents its own challenges, and although
several prior methods have studied related problems, they generally require a
discrete notion of tasks, with known ground-truth task boundaries. Such methods
typically adapt to each task in sequence, resetting the model between tasks,
rather than adapting continuously across tasks. In many real-world settings,
such discrete boundaries are unavailable, and may not even exist. To address
these settings, we propose a Fully Online Meta-Learning (FOML) algorithm, which
does not require any ground truth knowledge about the task boundaries and stays
fully online without resetting back to pre-trained weights. Our experiments
show that FOML was able to learn new tasks faster than the state-of-the-art
online learning methods on Rainbow-MNIST, CIFAR100 and CELEBA datasets.
Related papers
- MetaModulation: Learning Variational Feature Hierarchies for Few-Shot
Learning with Fewer Tasks [63.016244188951696]
We propose a method for few-shot learning with fewer tasks, which is by metaulation.
We modify parameters at various batch levels to increase the meta-training tasks.
We also introduce learning variational feature hierarchies by incorporating the variationalulation.
arXiv Detail & Related papers (2023-05-17T15:47:47Z) - Algorithm Design for Online Meta-Learning with Task Boundary Detection [63.284263611646]
We propose a novel algorithm for task-agnostic online meta-learning in non-stationary environments.
We first propose two simple but effective detection mechanisms of task switches and distribution shift.
We show that a sublinear task-averaged regret can be achieved for our algorithm under mild conditions.
arXiv Detail & Related papers (2023-02-02T04:02:49Z) - Dynamic Regret Analysis for Online Meta-Learning [0.0]
The online meta-learning framework has arisen as a powerful tool for the continual lifelong learning setting.
This formulation involves two levels: outer level which learns meta-learners and inner level which learns task-specific models.
We establish performance in terms of dynamic regret which handles changing environments from a global prospective.
We carry out our analyses in a setting, and in expectation prove a logarithmic local dynamic regret which explicitly depends on the total number of iterations.
arXiv Detail & Related papers (2021-09-29T12:12:59Z) - Meta-Reinforcement Learning by Tracking Task Non-stationarity [45.90345116853823]
We propose a novel algorithm (TRIO) that optimize for the future by explicitly tracking the task evolution through time.
Unlike most existing methods, TRIO does not assume Markovian task-evolution processes.
We evaluate our algorithm on different simulated problems and show it outperforms competitive baselines.
arXiv Detail & Related papers (2021-05-18T21:19:41Z) - Variable-Shot Adaptation for Online Meta-Learning [123.47725004094472]
We study the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.
We find that meta-learning solves the full task set with fewer overall labels and greater cumulative performance, compared to standard supervised methods.
These results suggest that meta-learning is an important ingredient for building learning systems that continuously learn and improve over a sequence of problems.
arXiv Detail & Related papers (2020-12-14T18:05:24Z) - Meta Automatic Curriculum Learning [35.13646854355393]
We introduce the concept of Meta-ACL, and formalize it in the context of black-box RL learners.
We present AGAIN, a first instantiation of Meta-ACL, and showcase its benefits for curriculum generation over classical ACL.
arXiv Detail & Related papers (2020-11-16T14:56:42Z) - Online Structured Meta-learning [137.48138166279313]
Current online meta-learning algorithms are limited to learn a globally-shared meta-learner.
We propose an online structured meta-learning (OSML) framework to overcome this limitation.
Experiments on three datasets demonstrate the effectiveness and interpretability of our proposed framework.
arXiv Detail & Related papers (2020-10-22T09:10:31Z) - Meta-learning the Learning Trends Shared Across Tasks [123.10294801296926]
Gradient-based meta-learning algorithms excel at quick adaptation to new tasks with limited data.
Existing meta-learning approaches only depend on the current task information during the adaptation.
We propose a 'Path-aware' model-agnostic meta-learning approach.
arXiv Detail & Related papers (2020-10-19T08:06:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.