Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation

• URL: http://arxiv.org/abs/2510.00212v1
• Date: Tue, 30 Sep 2025 19:42:15 GMT
• Title: Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation
• Authors: Yang Zhang, Huiwen Yan, Mushuang Liu,
• Abstract summary: We propose Directed-MAML, a novel task-directed meta-RL algorithm.<n>We show that Directed-MAML surpasses MAML-based baselines in computational efficiency and convergence speed.
• Score: 6.105325395844168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents notable challenges. First, MAML relies on second-order gradient computations, leading to significant computational and memory overhead. Second, the nested structure of optimization increases the problem's complexity, making convergence to a global optimum more challenging. To overcome these limitations, we propose Directed-MAML, a novel task-directed meta-RL algorithm. Before the second-order gradient step, Directed-MAML applies an additional first-order task-directed approximation to estimate the effect of second-order gradients, thereby accelerating convergence to the optimum and reducing computational cost. Experimental results demonstrate that Directed-MAML surpasses MAML-based baselines in computational efficiency and convergence speed in the scenarios of CartPole-v1, LunarLander-v2 and two-vehicle intersection crossing. Furthermore, we show that task-directed approximation can be effectively integrated into other meta-learning algorithms, such as First-Order Model-Agnostic Meta-Learning (FOMAML) and Meta Stochastic Gradient Descent(Meta-SGD), yielding improved computational efficiency and convergence speed.

Related papers

• Domain-Generalization to Improve Learning in Meta-Learning Algorithms [0.0]
  Domain Generalization Sharpness-Aware Minimization Model-Agnostic Meta-Learning (DGS-MAML)<n>This paper introduces Domain Generalization Sharpness-Aware Minimization Model-Agnostic Meta-Learning (DGS-MAML), a novel meta-learning algorithm designed to generalize across tasks with limited training data.<n> Experimental results on benchmark datasets show that DGS-MAML outperforms existing approaches in terms of accuracy and generalization.
  arXiv  Detail & Related papers  (2025-08-13T01:30:11Z)
• Model-Agnostic Multitask Fine-tuning for Few-shot Vision-Language Transfer Learning [59.38343286807997]
  We propose Model-Agnostic Multitask Fine-tuning (MAMF) for vision-language models on unseen tasks. Compared with model-agnostic meta-learning (MAML), MAMF discards the bi-level optimization and uses only first-order gradients. We show that MAMF consistently outperforms the classical fine-tuning method for few-shot transfer learning on five benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-09T17:26:53Z)
• MAML is a Noisy Contrastive Learner [72.04430033118426]
  Model-agnostic meta-learning (MAML) is one of the most popular and widely-adopted meta-learning algorithms nowadays. We provide a new perspective to the working mechanism of MAML and discover that: MAML is analogous to a meta-learner using a supervised contrastive objective function. We propose a simple but effective technique, zeroing trick, to alleviate such interference.
  arXiv  Detail & Related papers  (2021-06-29T12:52:26Z)
• Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation [19.792537914018933]
  Multi-task learning (MTL) aims to improve the generalization of several related tasks by learning them jointly. Modern meta-learning allows unseen tasks with limited labels during the test phase, in the hope of fast adaptation over them. We show that MTL shares the same optimization formulation with a class of gradient-based meta-learning (GBML) algorithms.
  arXiv  Detail & Related papers  (2021-06-16T17:58:23Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)
• On the Global Optimality of Model-Agnostic Meta-Learning [133.16370011229776]
  Model-a meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior. We characterize optimality of the stationary points attained by MAML for both learning and supervised learning, where the inner-level outer-level problems are solved via first-order optimization methods.
  arXiv  Detail & Related papers  (2020-06-23T17:33:14Z)
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters [152.03852111442114]
  Although model-agnostic metalearning (MAML) is a very successful algorithm meta-learning practice, it can have high computational complexity. Our paper shows that such complexity can significantly affect the overall convergence performance of ANIL.
  arXiv  Detail & Related papers  (2020-06-16T19:57:48Z)
• Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning [63.64636047748605]
  We develop a new theoretical framework to provide convergence guarantee for the general multi-step MAML algorithm. In particular, our results suggest that an inner-stage step needs to be chosen inversely proportional to $N$ of inner-stage steps in order for $N$ MAML to have guaranteed convergence.
  arXiv  Detail & Related papers  (2020-02-18T19:17:54Z)
• On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning [25.163423936635787]
  We consider Model-Agnostic Meta-Learning (MAML) methods for Reinforcement Learning (RL) problems. We propose a variant of the MAML method, named Gradient Meta-Reinforcement Learning (SG-MRL) We derive the iteration and sample complexity of SG-MRL to find an $ilon$-first-order stationary point, which, to the best of our knowledge, provides the first convergence guarantee for model-agnostic meta-reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2020-02-12T18:29:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.