Contextualizing Enhances Gradient Based Meta Learning

• URL: http://arxiv.org/abs/2007.10143v1
• Date: Fri, 17 Jul 2020 04:01:56 GMT
• Title: Contextualizing Enhances Gradient Based Meta Learning
• Authors: Evan Vogelbaum and Rumen Dangovski and Li Jing and Marin Solja\v{c}i\'c
• Abstract summary: We show how to equip meta learning methods with contextualizers and show that their use can significantly boost performance on a range of few shot learning datasets. Our approach is particularly apt for low-data environments where it is difficult to update parameters without overfitting.
• Score: 7.009032627535598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Meta learning methods have found success when applied to few shot classification problems, in which they quickly adapt to a small number of labeled examples. Prototypical representations, each representing a particular class, have been of particular importance in this setting, as they provide a compact form to convey information learned from the labeled examples. However, these prototypes are just one method of representing this information, and they are narrow in their scope and ability to classify unseen examples. We propose the implementation of contextualizers, which are generalizable prototypes that adapt to given examples and play a larger role in classification for gradient-based models. We demonstrate how to equip meta learning methods with contextualizers and show that their use can significantly boost performance on a range of few shot learning datasets. We also present figures of merit demonstrating the potential benefits of contextualizers, along with analysis of how models make use of them. Our approach is particularly apt for low-data environments where it is difficult to update parameters without overfitting. Our implementation and instructions to reproduce the experiments are available at https://github.com/naveace/proto-context.

Related papers

• Improve Meta-learning for Few-Shot Text Classification with All You Can Acquire from the Tasks [10.556477506959888]
  Existing methods often encounter difficulties in drawing accurate class prototypes from support set samples. Recent approaches attempt to incorporate external knowledge or pre-trained language models to augment data, but this requires additional resources. We propose a novel solution by adequately leveraging the information within the task itself.
  arXiv  Detail & Related papers  (2024-10-14T12:47:11Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• Not All Instances Contribute Equally: Instance-adaptive Class Representation Learning for Few-Shot Visual Recognition [94.04041301504567]
  Few-shot visual recognition refers to recognize novel visual concepts from a few labeled instances. We propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition.
  arXiv  Detail & Related papers  (2022-09-07T10:00:18Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Reordering Examples Helps during Priming-based Few-Shot Learning [6.579039107070663]
  We show that PERO can learn to generalize efficiently using as few as 10 examples. We demonstrate the effectiveness of the proposed method on the tasks of sentiment classification, natural language inference and fact retrieval.
  arXiv  Detail & Related papers  (2021-06-03T11:02:36Z)
• One Representation to Rule Them All: Identifying Out-of-Support Examples in Few-shot Learning with Generic Representations [0.8076739603800089]
  We describe a new method of identifying 'out-of-support' (OOS) examples within the Prototypical Networks framework. We show that our method outperforms other existing approaches in the literature.
  arXiv  Detail & Related papers  (2021-06-02T19:07:27Z)
• Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning [6.0158981171030685]
  Zero-shot learning aims to recognize instances of unseen classes, for which no visual instance is available during training. A good trade-off could be to employ short sentences in natural language as class descriptions. We show that while simple methods cannot achieve very good results with sentences alone, a combination of usual word embeddings and sentences can significantly outperform current state-of-the-art.
  arXiv  Detail & Related papers  (2020-10-06T18:22:21Z)
• Region Comparison Network for Interpretable Few-shot Image Classification [97.97902360117368]
  Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes. We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works. We also present a new way to generalize the interpretability from the level of tasks to categories.
  arXiv  Detail & Related papers  (2020-09-08T07:29:05Z)
• Few-shot Classification via Adaptive Attention [93.06105498633492]
  We propose a novel few-shot learning method via optimizing and fast adapting the query sample representation based on very few reference samples. As demonstrated experimentally, the proposed model achieves state-of-the-art classification results on various benchmark few-shot classification and fine-grained recognition datasets.
  arXiv  Detail & Related papers  (2020-08-06T05:52:59Z)
• UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation [52.487469544343305]
  Methods for object detection and segmentation rely on large scale instance-level annotations for training. We propose an intuitive and unified semi-supervised model that is applicable to a range of supervision.
  arXiv  Detail & Related papers  (2020-06-12T22:45:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.