One Line To Rule Them All: Generating LO-Shot Soft-Label Prototypes
- URL: http://arxiv.org/abs/2102.07834v1
- Date: Mon, 15 Feb 2021 20:21:29 GMT
- Title: One Line To Rule Them All: Generating LO-Shot Soft-Label Prototypes
- Authors: Ilia Sucholutsky, Nam-Hwui Kim, Ryan P. Browne, Matthias Schonlau
- Abstract summary: Prototype generation methods aim to create a small set of synthetic observations that accurately represent a training dataset.
assigning soft labels to prototypes can allow increasingly small sets of prototypes to accurately represent the original training dataset.
We propose a novel, modular method for generating soft-label lines that still maintains representational accuracy even when there are fewer prototypes than the number of classes in the data.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Increasingly large datasets are rapidly driving up the computational costs of
machine learning. Prototype generation methods aim to create a small set of
synthetic observations that accurately represent a training dataset but greatly
reduce the computational cost of learning from it. Assigning soft labels to
prototypes can allow increasingly small sets of prototypes to accurately
represent the original training dataset. Although foundational work on `less
than one'-shot learning has proven the theoretical plausibility of learning
with fewer than one observation per class, developing practical algorithms for
generating such prototypes remains an unexplored territory. We propose a novel,
modular method for generating soft-label prototypical lines that still
maintains representational accuracy even when there are fewer prototypes than
the number of classes in the data. In addition, we propose the Hierarchical
Soft-Label Prototype k-Nearest Neighbor classification algorithm based on these
prototypical lines. We show that our method maintains high classification
accuracy while greatly reducing the number of prototypes required to represent
a dataset, even when working with severely imbalanced and difficult data. Our
code is available at https://github.com/ilia10000/SLkNN.
Related papers
- Learning Prototype Classifiers for Long-Tailed Recognition [18.36167187657728]
We show that learning prototype classifiers addresses the biased softmax problem in long-tailed recognition.
We propose to jointly learn prototypes by using distances to prototypes in representation space as the logit scores for classification.
Our analysis shows that prototypes learned by Prototype classifiers are better separated than empirical centroids.
arXiv Detail & Related papers (2023-02-01T15:02:58Z) - Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV)
NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones.
We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
arXiv Detail & Related papers (2022-06-23T09:35:03Z) - Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
We present a nonparametric semantic segmentation model based on non-learnable prototypes.
Our framework yields compelling results over several datasets.
We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
arXiv Detail & Related papers (2022-03-28T21:15:32Z) - Prototype Memory for Large-scale Face Representation Learning [0.5524804393257919]
Softmax-based approach is not suitable for datasets with millions of persons.
We propose a novel face representation learning model called Prototype Memory.
We prove the effectiveness of the proposed model by extensive experiments on popular face recognition benchmarks.
arXiv Detail & Related papers (2021-05-05T15:08:34Z) - ALT-MAS: A Data-Efficient Framework for Active Testing of Machine
Learning Algorithms [58.684954492439424]
We propose a novel framework to efficiently test a machine learning model using only a small amount of labeled test data.
The idea is to estimate the metrics of interest for a model-under-test using Bayesian neural network (BNN)
arXiv Detail & Related papers (2021-04-11T12:14:04Z) - Optimal 1-NN Prototypes for Pathological Geometries [13.70633147306388]
Using prototype methods to reduce the size of training datasets can drastically reduce the computational cost of classification.
We show that it is difficult to find the optimal prototypes for a given dataset, and algorithms are used instead.
We propose an algorithm for finding nearly-optimal classifier prototypes in this setting, and use it to empirically validate the theoretical results.
arXiv Detail & Related papers (2020-10-31T10:15:08Z) - Prototype Completion with Primitive Knowledge for Few-Shot Learning [20.449056536438658]
Few-shot learning is a challenging task, which aims to learn a classifier for novel classes with few examples.
Pre-training based meta-learning methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning.
We propose a novel prototype completion based meta-learning framework.
arXiv Detail & Related papers (2020-09-10T16:09:34Z) - AutoSimulate: (Quickly) Learning Synthetic Data Generation [70.82315853981838]
We propose an efficient alternative for optimal synthetic data generation based on a novel differentiable approximation of the objective.
We demonstrate that the proposed method finds the optimal data distribution faster (up to $50times$), with significantly reduced training data generation (up to $30times$) and better accuracy ($+8.7%$) on real-world test datasets than previous methods.
arXiv Detail & Related papers (2020-08-16T11:36:11Z) - Learning Sparse Prototypes for Text Generation [120.38555855991562]
Prototype-driven text generation is inefficient at test time as a result of needing to store and index the entire training corpus.
We propose a novel generative model that automatically learns a sparse prototype support set that achieves strong language modeling performance.
In experiments, our model outperforms previous prototype-driven language models while achieving up to a 1000x memory reduction.
arXiv Detail & Related papers (2020-06-29T19:41:26Z) - Progressive Identification of True Labels for Partial-Label Learning [112.94467491335611]
Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label.
Most existing methods elaborately designed as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data.
This paper proposes a novel framework of classifier with flexibility on the model and optimization algorithm.
arXiv Detail & Related papers (2020-02-19T08:35:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.