TLXML: Task-Level Explanation of Meta-Learning via Influence Functions

• URL: http://arxiv.org/abs/2501.14271v2
• Date: Fri, 07 Feb 2025 18:37:21 GMT
• Title: TLXML: Task-Level Explanation of Meta-Learning via Influence Functions
• Authors: Yoshihiro Mitsuka, Shadan Golestan, Zahin Sufiyan, Sheila Schoepp, Shotaro Miwa, Osmar R. Zaiane,
• Abstract summary: We propose influence functions for explaining meta-learning that measure the sensitivities of training tasks to adaptation and inference.<n>We also argue that the approximation of the Hessian using the Gauss-Newton matrix resolves computational barriers peculiar to meta-learning.
• Score: 0.23090185577016445
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The scheme of adaptation via meta-learning is seen as an ingredient for solving the problem of data shortage or distribution shift in real-world applications, but it also brings the new risk of inappropriate updates of the model in the user environment, which increases the demand for explainability. Among the various types of XAI methods, establishing a method of explanation based on past experience in meta-learning requires special consideration due to its bi-level structure of training, which has been left unexplored. In this work, we propose influence functions for explaining meta-learning that measure the sensitivities of training tasks to adaptation and inference. We also argue that the approximation of the Hessian using the Gauss-Newton matrix resolves computational barriers peculiar to meta-learning. We demonstrate the adequacy of the method through experiments on task distinction and task distribution distinction using image classification tasks with MAML and Prototypical Network.

Related papers

• Meta-Learning and Meta-Reinforcement Learning - Tracing the Path towards DeepMind's Adaptive Agent [0.3906427348768226]
  Humans are highly effective at utilizing prior knowledge to adapt to novel tasks.<n>This survey provides a rigorous, task-based formalization of meta-learning and meta-reinforcement learning.
  arXiv  Detail & Related papers  (2026-02-23T13:39:58Z)
• Learning Task Representations from In-Context Learning [73.72066284711462]
  Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning. We introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads. We show that our method's effectiveness stems from aligning the distribution of the last hidden state with that of an optimally performing in-context-learned model.
  arXiv  Detail & Related papers  (2025-02-08T00:16:44Z)
• Rethinking Meta-Learning from a Learning Lens [17.00587250127854]
  We focus on the more fundamental learning to learn'' strategy of meta-learning to explore what causes errors and how to eliminate these errors without changing the environment. We propose using task relations to the optimization process of meta-learning and propose a plug-and-play method called Task Relation Learner (TRLearner) to achieve this goal.
  arXiv  Detail & Related papers  (2024-09-13T02:00:16Z)
• Unsupervised Meta-Learning via In-Context Learning [3.4165401459803335]
  We propose a novel approach to unsupervised meta-learning that leverages the generalization abilities of in-supervised learning. Our method reframes meta-learning as a sequence modeling problem, enabling the transformer encoder to learn task context from support images.
  arXiv  Detail & Related papers  (2024-05-25T08:29:46Z)
• MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Tasks [63.016244188951696]
  We propose a method for few-shot learning with fewer tasks, which is by metaulation. We modify parameters at various batch levels to increase the meta-training tasks. We also introduce learning variational feature hierarchies by incorporating the variationalulation.
  arXiv  Detail & Related papers  (2023-05-17T15:47:47Z)
• Algorithm Design for Online Meta-Learning with Task Boundary Detection [63.284263611646]
  We propose a novel algorithm for task-agnostic online meta-learning in non-stationary environments. We first propose two simple but effective detection mechanisms of task switches and distribution shift. We show that a sublinear task-averaged regret can be achieved for our algorithm under mild conditions.
  arXiv  Detail & Related papers  (2023-02-02T04:02:49Z)
• MetaNO: How to Transfer Your Knowledge on Learning Hidden Physics [39.83408993820245]
  We propose a novel meta-learning approach for neural operators, which can be seen as transferring the knowledge of solution operators between governing (unknown) PDEs with varying parameter fields. Our approach is a provably universal solution operator for multiple PDE solving tasks, with a key theoretical observation that underlying parameter fields can be captured in the first layer of neural operator models. As applications, we demonstrate the efficacy of our proposed approach on PDE-based datasets and a real-world material modeling problem, illustrating that our method can handle complex and nonlinear physical response learning tasks while greatly improving the sampling efficiency in unseen tasks.
  arXiv  Detail & Related papers  (2023-01-28T05:30:51Z)
• Set-based Meta-Interpolation for Few-Task Meta-Learning [79.4236527774689]
  We propose a novel domain-agnostic task augmentation method, Meta-Interpolation, to densify the meta-training task distribution. We empirically validate the efficacy of Meta-Interpolation on eight datasets spanning across various domains.
  arXiv  Detail & Related papers  (2022-05-20T06:53:03Z)
• Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation [87.98063273826702]
  We propose a memory imitation meta-learning (MemIML) method that enhances the model's reliance on support sets for task adaptation. A theoretical analysis is provided to prove the effectiveness of our method.
  arXiv  Detail & Related papers  (2022-03-22T12:41:55Z)
• Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
  Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios. By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels. Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
  arXiv  Detail & Related papers  (2021-06-04T20:15:34Z)
• On Data Efficiency of Meta-learning [17.739215706060605]
  We study the often overlooked aspect of the modern meta-learning algorithms -- their data efficiency. We introduce a new simple framework for evaluating meta-learning methods under a limit on the available supervision. We propose active meta-learning, which incorporates active data selection into learning-to-learn, leading to better performance of all methods in the limited supervision regime.
  arXiv  Detail & Related papers  (2021-01-30T01:44:12Z)
• Provable Meta-Learning of Linear Representations [114.656572506859]
  We provide fast, sample-efficient algorithms to address the dual challenges of learning a common set of features from multiple, related tasks, and transferring this knowledge to new, unseen tasks. We also provide information-theoretic lower bounds on the sample complexity of learning these linear features.
  arXiv  Detail & Related papers  (2020-02-26T18:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.