Informed Meta-Learning

• URL: http://arxiv.org/abs/2402.16105v4
• Date: Thu, 1 Aug 2024 09:53:03 GMT
• Title: Informed Meta-Learning
• Authors: Katarzyna Kobalczyk, Mihaela van der Schaar,
• Abstract summary: Meta-learning and informed ML stand out as two approaches for incorporating prior knowledge into ML pipelines. We formalise a hybrid paradigm, informed meta-learning, facilitating the incorporation of priors from unstructured knowledge representations. We demonstrate the potential benefits of informed meta-learning in improving data efficiency, robustness to observational noise and task distribution shifts.
• Score: 55.2480439325792
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In noisy and low-data regimes prevalent in real-world applications, a key challenge of machine learning lies in effectively incorporating inductive biases that promote data efficiency and robustness. Meta-learning and informed ML stand out as two approaches for incorporating prior knowledge into ML pipelines. While the former relies on a purely data-driven source of priors, the latter is guided by prior domain knowledge. In this paper, we formalise a hybrid paradigm, informed meta-learning, facilitating the incorporation of priors from unstructured knowledge representations, such as natural language; thus, unlocking complementarity in cross-task knowledge sharing of humans and machines. We establish the foundational components of informed meta-learning and present a concrete instantiation of this framework--the Informed Neural Process. Through a series of experiments, we demonstrate the potential benefits of informed meta-learning in improving data efficiency, robustness to observational noise and task distribution shifts.

Related papers

• Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning [79.46570165281084]
  We propose a Multi-Stage Knowledge Integration network (MulKI) to emulate the human learning process in distillation methods. MulKI achieves this through four stages, including Eliciting Ideas, Adding New Ideas, Distinguishing Ideas, and Making Connections. Our method demonstrates significant improvements in maintaining zero-shot capabilities while supporting continual learning across diverse downstream tasks.
  arXiv  Detail & Related papers  (2024-11-11T07:36:19Z)
• A Unified Framework for Continual Learning and Machine Unlearning [9.538733681436836]
  Continual learning and machine unlearning are crucial challenges in machine learning, typically addressed separately. We introduce a novel framework that jointly tackles both tasks by leveraging controlled knowledge distillation. Our approach enables efficient learning with minimal forgetting and effective targeted unlearning.
  arXiv  Detail & Related papers  (2024-08-21T06:49:59Z)
• Concept Discovery for Fast Adapatation [42.81705659613234]
  We introduce concept discovery to the few-shot learning problem, where we achieve more effective adaptation by meta-learning the structure among the data features. Our proposed method Concept-Based Model-Agnostic Meta-Learning (COMAML) has been shown to achieve consistent improvements in the structured data for both synthesized datasets and real-world datasets.
  arXiv  Detail & Related papers  (2023-01-19T02:33:58Z)
• Learning and Retrieval from Prior Data for Skill-based Imitation Learning [47.59794569496233]
  We develop a skill-based imitation learning framework that extracts temporally extended sensorimotor skills from prior data. We identify several key design choices that significantly improve performance on novel tasks.
  arXiv  Detail & Related papers  (2022-10-20T17:34:59Z)
• Learning with Limited Samples -- Meta-Learning and Applications to Communication Systems [46.760568562468606]
  Few-shot meta-learning optimize learning algorithms that can efficiently adapt to new tasks quickly. This review monograph provides an introduction to meta-learning by covering principles, algorithms, theory, and engineering applications.
  arXiv  Detail & Related papers  (2022-10-03T17:15:36Z)
• Knowledge-Aware Meta-learning for Low-Resource Text Classification [87.89624590579903]
  This paper studies a low-resource text classification problem and bridges the gap between meta-training and meta-testing tasks. We propose KGML to introduce additional representation for each sentence learned from the extracted sentence-specific knowledge graph.
  arXiv  Detail & Related papers  (2021-09-10T07:20:43Z)
• Online Structured Meta-learning [137.48138166279313]
  Current online meta-learning algorithms are limited to learn a globally-shared meta-learner. We propose an online structured meta-learning (OSML) framework to overcome this limitation. Experiments on three datasets demonstrate the effectiveness and interpretability of our proposed framework.
  arXiv  Detail & Related papers  (2020-10-22T09:10:31Z)
• Automated Relational Meta-learning [95.02216511235191]
  We propose an automated relational meta-learning framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph. We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-01-03T07:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.