Statistical Insight into Meta-Learning via Predictor Subspace Characterization and Quantification of Task Diversity

• URL: http://arxiv.org/abs/2509.18349v1
• Date: Mon, 22 Sep 2025 19:16:59 GMT
• Title: Statistical Insight into Meta-Learning via Predictor Subspace Characterization and Quantification of Task Diversity
• Authors: Saptati Datta, Nicolas W. Hengartner, Yulia Pimonova, Natalie E. Klein, Nicholas Lubbers,
• Abstract summary: We propose a framework for analyzing meta-learning through the lens of predictor subspace characterization and quantification of task diversity.<n>We provide both simulation-based and theoretical evidence indicating that achieving the desired prediction accuracy in meta-learning depends on the proportion of predictor variance aligned with the shared subspace.
• Score: 1.7942265700058986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-learning has emerged as a powerful paradigm for leveraging information across related tasks to improve predictive performance on new tasks. In this paper, we propose a statistical framework for analyzing meta-learning through the lens of predictor subspace characterization and quantification of task diversity. Specifically, we model the shared structure across tasks using a latent subspace and introduce a measure of diversity that captures heterogeneity across task-specific predictors. We provide both simulation-based and theoretical evidence indicating that achieving the desired prediction accuracy in meta-learning depends on the proportion of predictor variance aligned with the shared subspace, as well as on the accuracy of subspace estimation.

Related papers

• Meta-Statistical Learning: Supervised Learning of Statistical Inference [59.463430294611626]
  This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks.<n>We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems.
  arXiv  Detail & Related papers  (2025-02-17T18:04:39Z)
• Optimizing Multi-Task Learning for Accurate Spacecraft Pose Estimation [0.0]
  This paper explores the impact of different tasks within a multi-task learning framework for satellite pose estimation using monocular images. By integrating tasks such as direct pose estimation, keypoint prediction, object localization, and segmentation into a single network, the study aims to evaluate the reciprocal influence between tasks.
  arXiv  Detail & Related papers  (2024-10-16T15:44:15Z)
• LMD3: Language Model Data Density Dependence [78.76731603461832]
  We develop a methodology for analyzing language model task performance at the individual example level based on training data density estimation. Experiments with paraphrasing as a controlled intervention on finetuning data demonstrate that increasing the support in the training distribution for specific test queries results in a measurable increase in density. We conclude that our framework can provide statistical evidence of the dependence of a target model's predictions on subsets of its training data.
  arXiv  Detail & Related papers  (2024-05-10T09:03:27Z)
• On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training [72.8087629914444]
  We study the impact of the trade-off between the intra-class diversity (the number of samples per class) and the inter-class diversity (the number of classes) of a supervised pre-training dataset. With the size of the pre-training dataset fixed, the best downstream performance comes with a balance on the intra-/inter-class diversity.
  arXiv  Detail & Related papers  (2023-05-20T16:23:50Z)
• MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
  We present a novel approach to extracting inductive biases from a set of related datasets. We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space. Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
  arXiv  Detail & Related papers  (2022-10-24T15:14:26Z)
• Probabilistic task modelling for meta-learning [44.072592379328036]
  We propose a generative probabilistic model for collections of tasks used in meta-learning. The proposed model combines variational auto-encoding and latent Dirichlet allocation to model each task as a mixture of Gaussian distribution in an embedding space.
  arXiv  Detail & Related papers  (2021-06-09T04:34:12Z)
• Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
  Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios. By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels. Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
  arXiv  Detail & Related papers  (2021-06-04T20:15:34Z)
• An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning [44.320945743871285]
  We present novel information-theoretic bounds on the average absolute value of the meta-generalization gap. Our bounds explicitly capture the impact of task relatedness, the number of tasks, and the number of data samples per task on the meta-generalization gap.
  arXiv  Detail & Related papers  (2021-01-21T01:38:16Z)
• Meta-learning framework with applications to zero-shot time-series forecasting [82.61728230984099]
  This work provides positive evidence using a broad meta-learning framework. residual connections act as a meta-learning adaptation mechanism. We show that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining.
  arXiv  Detail & Related papers  (2020-02-07T16:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.