Related papers: Optimistic Meta-Gradients

Optimistic Meta-Gradients

URL: http://arxiv.org/abs/2301.03236v1
Date: Mon, 9 Jan 2023 10:05:12 GMT
Title: Optimistic Meta-Gradients
Authors: Sebastian Flennerhag and Tom Zahavy and Brendan O'Donoghue and Hado van Hasselt and Andr\'as Gy\"orgy and Satinder Singh
Abstract summary: We study the connection between gradient-based meta-learning and convex op-timisation. We show that optimism in meta-learning can be captured through Bootstrapped Meta-Gradients.
Score: 37.11276919046808
License: http://creativecommons.org/licenses/by/4.0/
Abstract: We study the connection between gradient-based meta-learning and convex op-timisation. We observe that gradient descent with momentum is a special case of meta-gradients, and building on recent results in optimisation, we prove convergence rates for meta-learning in the single task setting. While a meta-learned update rule can yield faster convergence up to constant factor, it is not sufficient for acceleration. Instead, some form of optimism is required. We show that optimism in meta-learning can be captured through Bootstrapped Meta-Gradients (Flennerhag et al., 2022), providing deeper insight into its underlying mechanics.

Related papers

Arbitrary Order Meta-Learning with Simple Population-Based Evolution [1.6114012813668934]
We show that simple population-based evolution implicitly optimises for arbitrarily-high order meta- parameters. We then introduce a minimal self-referential parameterisation, which in principle enables arbitrary-order meta-learning.
arXiv Detail & Related papers (2023-03-16T16:55:26Z)
Meta Mirror Descent: Optimiser Learning for Fast Convergence [85.98034682899855]
We take a different perspective starting from mirror descent rather than gradient descent, and meta-learning the corresponding Bregman divergence. Within this paradigm, we formalise a novel meta-learning objective of minimising the regret bound of learning. Unlike many meta-learned optimisers, it also supports convergence and generalisation guarantees and uniquely does so without requiring validation data.
arXiv Detail & Related papers (2022-03-05T11:41:13Z)
One Step at a Time: Pros and Cons of Multi-Step Meta-Gradient Reinforcement Learning [61.662504399411695]
We introduce a novel method mixing multiple inner steps that enjoys a more accurate and robust meta-gradient signal. When applied to the Snake game, the mixing meta-gradient algorithm can cut the variance by a factor of 3 while achieving similar or higher performance.
arXiv Detail & Related papers (2021-10-30T08:36:52Z)
Bootstrapped Meta-Learning [48.017607959109924]
We propose an algorithm that tackles a challenging meta-optimisation problem by letting the meta-learner teach itself. The algorithm first bootstraps a target from the meta-learner, then optimises the meta-learner by minimising the distance to that target under a chosen (pseudo-)metric. We achieve a new state-of-the art for model-free agents on the Atari ALE benchmark, improve upon MAML in few-shot learning, and demonstrate how our approach opens up new possibilities.
arXiv Detail & Related papers (2021-09-09T18:29:05Z)
Faster Meta Update Strategy for Noise-Robust Deep Learning [62.08964100618873]
We introduce a novel Faster Meta Update Strategy (FaMUS) to replace the most expensive step in the meta gradient with a faster layer-wise approximation. We show our method is able to save two-thirds of the training time while still maintaining the comparable or achieving even better generalization performance.
arXiv Detail & Related papers (2021-04-30T16:19:07Z)
Large-Scale Meta-Learning with Continual Trajectory Shifting [76.29017270864308]
We show that allowing the meta-learners to take a larger number of inner gradient steps better captures the structure of heterogeneous and large-scale tasks. In order to increase the frequency of meta-updates, we propose to estimate the required shift of the task-specific parameters. We show that the algorithm largely outperforms the previous first-order meta-learning methods in terms of both generalization performance and convergence.
arXiv Detail & Related papers (2021-02-14T18:36:33Z)
Guarantees for Tuning the Step Size using a Learning-to-Learn Approach [18.838453594698166]
We give meta-optimization guarantees for the learning-to-learn approach on a simple problem of tuning the step size for quadratic loss. Although there is a way to design the meta-objective so that the meta-gradient remains bounded, computing the meta-gradient directly using backpropagation leads to numerical issues. We also characterize when it is necessary to compute the meta-objective on a separate validation set to ensure the performance of the learned.
arXiv Detail & Related papers (2020-06-30T02:59:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.