Bootstrapped Meta-Learning

• URL: http://arxiv.org/abs/2109.04504v1
• Date: Thu, 9 Sep 2021 18:29:05 GMT
• Title: Bootstrapped Meta-Learning
• Authors: Sebastian Flennerhag and Yannick Schroecker and Tom Zahavy and Hado van Hasselt and David Silver and Satinder Singh
• Abstract summary: We propose an algorithm that tackles a challenging meta-optimisation problem by letting the meta-learner teach itself. The algorithm first bootstraps a target from the meta-learner, then optimises the meta-learner by minimising the distance to that target under a chosen (pseudo-)metric. We achieve a new state-of-the art for model-free agents on the Atari ALE benchmark, improve upon MAML in few-shot learning, and demonstrate how our approach opens up new possibilities.
• Score: 48.017607959109924
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-learning empowers artificial intelligence to increase its efficiency by learning how to learn. Unlocking this potential involves overcoming a challenging meta-optimisation problem that often exhibits ill-conditioning, and myopic meta-objectives. We propose an algorithm that tackles these issues by letting the meta-learner teach itself. The algorithm first bootstraps a target from the meta-learner, then optimises the meta-learner by minimising the distance to that target under a chosen (pseudo-)metric. Focusing on meta-learning with gradients, we establish conditions that guarantee performance improvements and show that the improvement is related to the target distance. Thus, by controlling curvature, the distance measure can be used to ease meta-optimization, for instance by reducing ill-conditioning. Further, the bootstrapping mechanism can extend the effective meta-learning horizon without requiring backpropagation through all updates. The algorithm is versatile and easy to implement. We achieve a new state-of-the art for model-free agents on the Atari ALE benchmark, improve upon MAML in few-shot learning, and demonstrate how our approach opens up new possibilities by meta-learning efficient exploration in a Q-learning agent.

Related papers

• ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning [49.447777286862994]
  ConML is a universal meta-learning framework that can be applied to various meta-learning algorithms. We demonstrate that ConML integrates seamlessly with optimization-based, metric-based, and amortization-based meta-learning algorithms.
  arXiv  Detail & Related papers  (2024-10-08T12:22:10Z)
• Eliminating Meta Optimization Through Self-Referential Meta Learning [5.584060970507506]
  We investigate self-referential meta learning systems that modify themselves without the need for explicit meta optimization. A neural network self-modifies to solve bandit and classic control tasks, improves its self-modifications, and learns how to learn.
  arXiv  Detail & Related papers  (2022-12-29T17:53:40Z)
• Meta-Learning with Self-Improving Momentum Target [72.98879709228981]
  We propose Self-improving Momentum Target (SiMT) to improve the performance of a meta-learner. SiMT generates the target model by adapting from the temporal ensemble of the meta-learner. We show that SiMT brings a significant performance gain when combined with a wide range of meta-learning methods.
  arXiv  Detail & Related papers  (2022-10-11T06:45:15Z)
• Meta Mirror Descent: Optimiser Learning for Fast Convergence [85.98034682899855]
  We take a different perspective starting from mirror descent rather than gradient descent, and meta-learning the corresponding Bregman divergence. Within this paradigm, we formalise a novel meta-learning objective of minimising the regret bound of learning. Unlike many meta-learned optimisers, it also supports convergence and generalisation guarantees and uniquely does so without requiring validation data.
  arXiv  Detail & Related papers  (2022-03-05T11:41:13Z)
• Accelerating Gradient-based Meta Learner [2.1349209400003932]
  We propose various acceleration techniques to speed up meta learning algorithms such as MAML (Model Agnostic Meta Learning) We introduce a novel method of training tasks in clusters, which not only accelerates the meta learning process but also improves model accuracy performance.
  arXiv  Detail & Related papers  (2021-10-27T14:27:36Z)
• Meta-Gradient Reinforcement Learning with an Objective Discovered Online [54.15180335046361]
  We propose an algorithm based on meta-gradient descent that discovers its own objective, flexibly parameterised by a deep neural network. Because the objective is discovered online, it can adapt to changes over time. On the Atari Learning Environment, the meta-gradient algorithm adapts over time to learn with greater efficiency.
  arXiv  Detail & Related papers  (2020-07-16T16:17:09Z)
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach [18.838453594698166]
  We give meta-optimization guarantees for the learning-to-learn approach on a simple problem of tuning the step size for quadratic loss. Although there is a way to design the meta-objective so that the meta-gradient remains bounded, computing the meta-gradient directly using backpropagation leads to numerical issues. We also characterize when it is necessary to compute the meta-objective on a separate validation set to ensure the performance of the learned.
  arXiv  Detail & Related papers  (2020-06-30T02:59:35Z)
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning [0.0]
  Meta-learning also known as learning-to-learn which seeks rapid and accurate model adaptation to unseen tasks. We briefly introduce meta-learning methodologies in the following categories: black-box meta-learning, metric-based meta-learning, layered meta-learning and Bayesian meta-learning framework.
  arXiv  Detail & Related papers  (2020-04-17T03:11:08Z)
• Incremental Object Detection via Meta-Learning [77.55310507917012]
  We propose a meta-learning approach that learns to reshape model gradients, such that information across incremental tasks is optimally shared. In comparison to existing meta-learning methods, our approach is task-agnostic, allows incremental addition of new-classes and scales to high-capacity models for object detection.
  arXiv  Detail & Related papers  (2020-03-17T13:40:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.