Online Structured Meta-learning

• URL: http://arxiv.org/abs/2010.11545v1
• Date: Thu, 22 Oct 2020 09:10:31 GMT
• Title: Online Structured Meta-learning
• Authors: Huaxiu Yao, Yingbo Zhou, Mehrdad Mahdavi, Zhenhui Li, Richard Socher, Caiming Xiong
• Abstract summary: Current online meta-learning algorithms are limited to learn a globally-shared meta-learner. We propose an online structured meta-learning (OSML) framework to overcome this limitation. Experiments on three datasets demonstrate the effectiveness and interpretability of our proposed framework.
• Score: 137.48138166279313
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning quickly is of great importance for machine intelligence deployed in online platforms. With the capability of transferring knowledge from learned tasks, meta-learning has shown its effectiveness in online scenarios by continuously updating the model with the learned prior. However, current online meta-learning algorithms are limited to learn a globally-shared meta-learner, which may lead to sub-optimal results when the tasks contain heterogeneous information that are distinct by nature and difficult to share. We overcome this limitation by proposing an online structured meta-learning (OSML) framework. Inspired by the knowledge organization of human and hierarchical feature representation, OSML explicitly disentangles the meta-learner as a meta-hierarchical graph with different knowledge blocks. When a new task is encountered, it constructs a meta-knowledge pathway by either utilizing the most relevant knowledge blocks or exploring new blocks. Through the meta-knowledge pathway, the model is able to quickly adapt to the new task. In addition, new knowledge is further incorporated into the selected blocks. Experiments on three datasets demonstrate the effectiveness and interpretability of our proposed framework in the context of both homogeneous and heterogeneous tasks.

Related papers

• ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning [49.447777286862994]
  ConML is a universal meta-learning framework that can be applied to various meta-learning algorithms. We demonstrate that ConML integrates seamlessly with optimization-based, metric-based, and amortization-based meta-learning algorithms.
  arXiv  Detail & Related papers  (2024-10-08T12:22:10Z)
• When Meta-Learning Meets Online and Continual Learning: A Survey [39.53836535326121]
  meta-learning is a data-driven approach to optimize the learning algorithm. Continual learning and online learning, both of which involve incrementally updating a model with streaming data. This paper organizes various problem settings using consistent terminology and formal descriptions.
  arXiv  Detail & Related papers  (2023-11-09T09:49:50Z)
• Concept Discovery for Fast Adapatation [42.81705659613234]
  We introduce concept discovery to the few-shot learning problem, where we achieve more effective adaptation by meta-learning the structure among the data features. Our proposed method Concept-Based Model-Agnostic Meta-Learning (COMAML) has been shown to achieve consistent improvements in the structured data for both synthesized datasets and real-world datasets.
  arXiv  Detail & Related papers  (2023-01-19T02:33:58Z)
• Fully Online Meta-Learning Without Task Boundaries [80.09124768759564]
  We study how meta-learning can be applied to tackle online problems of this nature. We propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries. Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods.
  arXiv  Detail & Related papers  (2022-02-01T07:51:24Z)
• Knowledge-Aware Meta-learning for Low-Resource Text Classification [87.89624590579903]
  This paper studies a low-resource text classification problem and bridges the gap between meta-training and meta-testing tasks. We propose KGML to introduce additional representation for each sentence learned from the extracted sentence-specific knowledge graph.
  arXiv  Detail & Related papers  (2021-09-10T07:20:43Z)
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning [0.0]
  Meta-learning also known as learning-to-learn which seeks rapid and accurate model adaptation to unseen tasks. We briefly introduce meta-learning methodologies in the following categories: black-box meta-learning, metric-based meta-learning, layered meta-learning and Bayesian meta-learning framework.
  arXiv  Detail & Related papers  (2020-04-17T03:11:08Z)
• Revisiting Meta-Learning as Supervised Learning [69.2067288158133]
  We aim to provide a principled, unifying framework by revisiting and strengthening the connection between meta-learning and traditional supervised learning. By treating pairs of task-specific data sets and target models as (feature, label) samples, we can reduce many meta-learning algorithms to instances of supervised learning. This view not only unifies meta-learning into an intuitive and practical framework but also allows us to transfer insights from supervised learning directly to improve meta-learning.
  arXiv  Detail & Related papers  (2020-02-03T06:13:01Z)
• Automated Relational Meta-learning [95.02216511235191]
  We propose an automated relational meta-learning framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph. We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-01-03T07:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.