Learning Prototype-oriented Set Representations for Meta-Learning
- URL: http://arxiv.org/abs/2110.09140v1
- Date: Mon, 18 Oct 2021 09:49:05 GMT
- Title: Learning Prototype-oriented Set Representations for Meta-Learning
- Authors: Dandan Guo, Long Tian, Minghe Zhang, Mingyuan Zhou, Hongyuan Zha
- Abstract summary: Learning from set-structured data is a fundamental problem that has recently attracted increasing attention.
This paper provides a novel optimal transport based way to improve existing summary networks.
We further instantiate it to the cases of few-shot classification and implicit meta generative modeling.
- Score: 85.19407183975802
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning from set-structured data is a fundamental problem that has recently
attracted increasing attention, where a series of summary networks are
introduced to deal with the set input. In fact, many meta-learning problems can
be treated as set-input tasks. Most existing summary networks aim to design
different architectures for the input set in order to enforce permutation
invariance. However, scant attention has been paid to the common cases where
different sets in a meta-distribution are closely related and share certain
statistical properties. Viewing each set as a distribution over a set of global
prototypes, this paper provides a novel optimal transport (OT) based way to
improve existing summary networks. To learn the distribution over the global
prototypes, we minimize its OT distance to the set empirical distribution over
data points, providing a natural unsupervised way to improve the summary
network. Since our plug-and-play framework can be applied to many meta-learning
problems, we further instantiate it to the cases of few-shot classification and
implicit meta generative modeling. Extensive experiments demonstrate that our
framework significantly improves the existing summary networks on learning more
powerful summary statistics from sets and can be successfully integrated into
metric-based few-shot classification and generative modeling applications,
providing a promising tool for addressing set-input and meta-learning problems.
Related papers
- Self-Supervised Representation Learning with Meta Comprehensive
Regularization [11.387994024747842]
We introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks.
We update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features.
We provide theoretical support for our proposed method from information theory and causal counterfactual perspective.
arXiv Detail & Related papers (2024-03-03T15:53:48Z) - Improving Meta-Learning Generalization with Activation-Based
Early-Stopping [12.299371455015239]
Meta-Learning algorithms for few-shot learning aim to train neural networks capable of generalizing to novel tasks using only a few examples.
Early-stopping is critical for performance, halting model training when it reaches optimal generalization to the new task distribution.
This is problematic in few-shot transfer learning settings, where the meta-test set comes from a different target dataset.
arXiv Detail & Related papers (2022-08-03T22:55:45Z) - Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios.
By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels.
Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
arXiv Detail & Related papers (2021-06-04T20:15:34Z) - Set Representation Learning with Generalized Sliced-Wasserstein
Embeddings [22.845403993200932]
We propose a geometrically-interpretable framework for learning representations from set-structured data.
In particular, we treat elements of a set as samples from a probability measure and propose an exact Euclidean embedding for Generalized Sliced Wasserstein.
We evaluate our proposed framework on multiple supervised and unsupervised set learning tasks and demonstrate its superiority over state-of-the-art set representation learning approaches.
arXiv Detail & Related papers (2021-03-05T19:00:34Z) - Adaptive Prototypical Networks with Label Words and Joint Representation
Learning for Few-Shot Relation Classification [17.237331828747006]
This work focuses on few-shot relation classification (FSRC)
We propose an adaptive mixture mechanism to add label words to the representation of the class prototype.
Experiments have been conducted on FewRel under different few-shot (FS) settings.
arXiv Detail & Related papers (2021-01-10T11:25:42Z) - Variable-Shot Adaptation for Online Meta-Learning [123.47725004094472]
We study the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.
We find that meta-learning solves the full task set with fewer overall labels and greater cumulative performance, compared to standard supervised methods.
These results suggest that meta-learning is an important ingredient for building learning systems that continuously learn and improve over a sequence of problems.
arXiv Detail & Related papers (2020-12-14T18:05:24Z) - Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn.
We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
arXiv Detail & Related papers (2020-12-01T11:46:03Z) - Few-Shot Unsupervised Continual Learning through Meta-Examples [21.954394608030388]
We introduce a novel and complex setting involving unsupervised meta-continual learning with unbalanced tasks.
We exploit a meta-learning scheme that simultaneously alleviates catastrophic forgetting and favors the generalization to new tasks.
Experimental results on few-shot learning benchmarks show competitive performance even compared to the supervised case.
arXiv Detail & Related papers (2020-09-17T07:02:07Z) - Improving Generalization in Meta-learning via Task Augmentation [69.83677015207527]
We propose two task augmentation methods, including MetaMix and Channel Shuffle.
Both MetaMix and Channel Shuffle outperform state-of-the-art results by a large margin across many datasets.
arXiv Detail & Related papers (2020-07-26T01:50:42Z) - Meta-learning framework with applications to zero-shot time-series
forecasting [82.61728230984099]
This work provides positive evidence using a broad meta-learning framework.
residual connections act as a meta-learning adaptation mechanism.
We show that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining.
arXiv Detail & Related papers (2020-02-07T16:39:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.