Binomial Gradient-Based Meta-Learning for Enhanced Meta-Gradient Estimation

• URL: http://arxiv.org/abs/2604.13263v1
• Date: Tue, 14 Apr 2026 19:52:15 GMT
• Title: Binomial Gradient-Based Meta-Learning for Enhanced Meta-Gradient Estimation
• Abstract summary: Gradient-based meta-learning (GBML) relies on gradient descent to adapt the prior to a new task.<n>Existing methods approximate the gradient of prior parameters (meta-gradient) via truncated backpropagation, yet suffer large approximation errors.<n>This work puts forth binomial GBML (BinomGBML), which relies on a truncated binomial expansion for meta-gradient estimation.
• Score: 37.795850351021016
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-learning offers a principled framework leveraging \emph{task-invariant} priors from related tasks, with which \emph{task-specific} models can be fine-tuned on downstream tasks, even with limited data records. Gradient-based meta-learning (GBML) relies on gradient descent (GD) to adapt the prior to a new task. Albeit effective, these methods incur high computational overhead that scales linearly with the number of GD steps. To enhance efficiency and scalability, existing methods approximate the gradient of prior parameters (meta-gradient) via truncated backpropagation, yet suffer large approximation errors. Targeting accurate approximation, this work puts forth binomial GBML (BinomGBML), which relies on a truncated binomial expansion for meta-gradient estimation. This novel expansion endows more information in the meta-gradient estimation via efficient parallel computation. As a running paradigm applied to model-agnostic meta-learning (MAML), the resultant BinomMAML provably enjoys error bounds that not only improve upon existing approaches, but also decay super-exponentially under mild conditions. Numerical tests corroborate the theoretical analysis and showcase boosted performance with slightly increased computational overhead.

Related papers

• Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation [6.105325395844168]
  We propose Directed-MAML, a novel task-directed meta-RL algorithm.<n>We show that Directed-MAML surpasses MAML-based baselines in computational efficiency and convergence speed.
  arXiv  Detail & Related papers  (2025-09-30T19:42:15Z)
• Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning [41.28925127311434]
  meta-learning is a principled approach to learning a new task with limited data records.<n>We present a novel method for preconditioning that speeds up convergence of the per-task training.<n>Tests on few-shot learning datasets demonstrate the superior empirical performance of the novel algorithm.
  arXiv  Detail & Related papers  (2025-09-02T15:23:21Z)
• Theoretical Characterization of the Generalization Performance of Overfitted Meta-Learning [70.52689048213398]
  This paper studies the performance of overfitted meta-learning under a linear regression model with Gaussian features. We find new and interesting properties that do not exist in single-task linear regression. Our analysis suggests that benign overfitting is more significant and easier to observe when the noise and the diversity/fluctuation of the ground truth of each training task are large.
  arXiv  Detail & Related papers  (2023-04-09T20:36:13Z)
• Scalable Bayesian Meta-Learning through Generalized Implicit Gradients [64.21628447579772]
  Implicit Bayesian meta-learning (iBaML) method broadens the scope of learnable priors, but also quantifies the associated uncertainty. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one.
  arXiv  Detail & Related papers  (2023-03-31T02:10:30Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• Continuous-Time Meta-Learning with Forward Mode Differentiation [65.26189016950343]
  We introduce Continuous Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of a gradient vector field. Treating the learning process as an ODE offers the notable advantage that the length of the trajectory is now continuous. We show empirically its efficiency in terms of runtime and memory usage, and we illustrate its effectiveness on a range of few-shot image classification problems.
  arXiv  Detail & Related papers  (2022-03-02T22:35:58Z)
• Meta-Learning with Adjoint Methods [16.753336086160598]
  A Meta-Learning (MAML) is widely used to find a good initialization for a family of tasks. Despite its success, a critical challenge in MAML is to calculate the gradient w.r.t the initialization of a long training trajectory for the sampled tasks. We propose Adjoint MAML (A-MAML) to address this problem. We demonstrate the advantage of our approach in both synthetic and real-world meta-learning tasks.
  arXiv  Detail & Related papers  (2021-10-16T01:18:50Z)
• B-SMALL: A Bayesian Neural Network approach to Sparse Model-Agnostic Meta-Learning [2.9189409618561966]
  We propose a Bayesian neural network based MAML algorithm, which we refer to as the B-SMALL algorithm. We demonstrate the performance of B-MAML using classification and regression tasks, and highlight that training a sparsifying BNN using MAML indeed improves the parameter footprint of the model.
  arXiv  Detail & Related papers  (2021-01-01T09:19:48Z)
• Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning [63.64636047748605]
  We develop a new theoretical framework to provide convergence guarantee for the general multi-step MAML algorithm. In particular, our results suggest that an inner-stage step needs to be chosen inversely proportional to $N$ of inner-stage steps in order for $N$ MAML to have guaranteed convergence.
  arXiv  Detail & Related papers  (2020-02-18T19:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.