Variational Metric Scaling for Metric-Based Meta-Learning
- URL: http://arxiv.org/abs/1912.11809v2
- Date: Wed, 26 Aug 2020 10:07:54 GMT
- Title: Variational Metric Scaling for Metric-Based Meta-Learning
- Authors: Jiaxin Chen, Li-Ming Zhan, Xiao-Ming Wu, Fu-lai Chung
- Abstract summary: We recast metric-based meta-learning from a prototypical perspective and develop a variational metric scaling framework.
Our method is end-to-end without any pre-training and can be used as a simple plug-and-play module for existing metric-based meta-algorithms.
- Score: 37.392840869320686
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Metric-based meta-learning has attracted a lot of attention due to its
effectiveness and efficiency in few-shot learning. Recent studies show that
metric scaling plays a crucial role in the performance of metric-based
meta-learning algorithms. However, there still lacks a principled method for
learning the metric scaling parameter automatically. In this paper, we recast
metric-based meta-learning from a Bayesian perspective and develop a
variational metric scaling framework for learning a proper metric scaling
parameter. Firstly, we propose a stochastic variational method to learn a
single global scaling parameter. To better fit the embedding space to a given
data distribution, we extend our method to learn a dimensional scaling vector
to transform the embedding space. Furthermore, to learn task-specific
embeddings, we generate task-dependent dimensional scaling vectors with
amortized variational inference. Our method is end-to-end without any
pre-training and can be used as a simple plug-and-play module for existing
metric-based meta-algorithms. Experiments on mini-ImageNet show that our
methods can be used to consistently improve the performance of existing
metric-based meta-algorithms including prototypical networks and TADAM. The
source code can be downloaded from
https://github.com/jiaxinchen666/variational-scaling.
Related papers
- Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval [37.601607544184915]
Metric learning aims to build a distance metric typically by learning an effective embedding function that maps similar objects into nearby points.
Despite recent advances in deep metric learning, it remains challenging for the learned metric to generalize to unseen classes with a substantial domain gap.
We propose a new problem of few-shot metric learning that aims to adapt the embedding function to the target domain with only a few annotated data.
arXiv Detail & Related papers (2022-11-14T05:10:17Z) - Learn to Learn Metric Space for Few-Shot Segmentation of 3D Shapes [17.217954254022573]
We introduce a meta-learning-based method for few-shot 3D shape segmentation where only a few labeled samples are provided for the unseen classes.
We demonstrate the superior performance of our proposed on the ShapeNet part dataset under the few-shot scenario, compared with well-established baseline and state-of-the-art semi-supervised methods.
arXiv Detail & Related papers (2021-07-07T01:47:00Z) - Finding Significant Features for Few-Shot Learning using Dimensionality
Reduction [0.0]
This module helps to improve the accuracy performance by allowing the similarity function, given by the metric learning method, to have more discriminative features for the classification.
Our method outperforms the metric learning baselines in the miniImageNet dataset by around 2% in accuracy performance.
arXiv Detail & Related papers (2021-07-06T16:36:57Z) - Learning an Explicit Hyperparameter Prediction Function Conditioned on
Tasks [62.63852372239708]
Meta learning aims to learn the learning methodology for machine learning from observed tasks, so as to generalize to new query tasks.
We interpret such learning methodology as learning an explicit hyper- parameter prediction function shared by all training tasks.
Such setting guarantees that the meta-learned learning methodology is able to flexibly fit diverse query tasks.
arXiv Detail & Related papers (2021-07-06T04:05:08Z) - Meta-Generating Deep Attentive Metric for Few-shot Classification [53.07108067253006]
We present a novel deep metric meta-generation method to generate a specific metric for a new few-shot learning task.
In this study, we structure the metric using a three-layer deep attentive network that is flexible enough to produce a discriminative metric for each task.
We gain surprisingly obvious performance improvement over state-of-the-art competitors, especially in the challenging cases.
arXiv Detail & Related papers (2020-12-03T02:07:43Z) - Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
We introduce a fast optimization-based meta-learning method for few-shot classification.
Our strategy enables important aspects of the base learner objective to be learned during meta-training.
We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
arXiv Detail & Related papers (2020-10-01T15:59:31Z) - Meta-Learning Symmetries by Reparameterization [63.85144439337671]
We present a method for learning and encoding equivariances into networks by learning corresponding parameter sharing patterns from data.
Our experiments suggest that it can automatically learn to encode equivariances to common transformations used in image processing tasks.
arXiv Detail & Related papers (2020-07-06T17:59:54Z) - Supervised Categorical Metric Learning with Schatten p-Norms [10.995886294197412]
We propose a method, called CPML for emphcategorical projected metric learning, to address the problem of metric learning in categorical data.
We make use of the Value Distance Metric to represent our data and propose new distances based on this representation.
We then show how to efficiently learn new metrics.
arXiv Detail & Related papers (2020-02-26T01:17:12Z) - Incremental Meta-Learning via Indirect Discriminant Alignment [118.61152684795178]
We develop a notion of incremental learning during the meta-training phase of meta-learning.
Our approach performs favorably at test time as compared to training a model with the full meta-training set.
arXiv Detail & Related papers (2020-02-11T01:39:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.