Active metric learning and classification using similarity queries
- URL: http://arxiv.org/abs/2202.01953v1
- Date: Fri, 4 Feb 2022 03:34:29 GMT
- Title: Active metric learning and classification using similarity queries
- Authors: Namrata Nadagouda, Austin Xu and Mark A. Davenport
- Abstract summary: We show that a novel unified query framework can be applied to any problem in which a key component is learning a representation of the data that reflects similarity.
We demonstrate the effectiveness of the proposed strategy on two tasks -- active metric learning and active classification.
- Score: 21.589707834542338
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Active learning is commonly used to train label-efficient models by
adaptively selecting the most informative queries. However, most active
learning strategies are designed to either learn a representation of the data
(e.g., embedding or metric learning) or perform well on a task (e.g.,
classification) on the data. However, many machine learning tasks involve a
combination of both representation learning and a task-specific goal. Motivated
by this, we propose a novel unified query framework that can be applied to any
problem in which a key component is learning a representation of the data that
reflects similarity. Our approach builds on similarity or nearest neighbor (NN)
queries which seek to select samples that result in improved embeddings. The
queries consist of a reference and a set of objects, with an oracle selecting
the object most similar (i.e., nearest) to the reference. In order to reduce
the number of solicited queries, they are chosen adaptively according to an
information theoretic criterion. We demonstrate the effectiveness of the
proposed strategy on two tasks -- active metric learning and active
classification -- using a variety of synthetic and real world datasets. In
particular, we demonstrate that actively selected NN queries outperform
recently developed active triplet selection methods in a deep metric learning
setting. Further, we show that in classification, actively selecting class
labels can be reformulated as a process of selecting the most informative NN
query, allowing direct application of our method.
Related papers
- Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars [66.823588073584]
Large language models (LLMs) have shown impressive capabilities in real-world applications.
The quality of these exemplars in the prompt greatly impacts performance.
Existing methods fail to adequately account for the impact of exemplar ordering on the performance.
arXiv Detail & Related papers (2024-05-25T08:23:05Z) - Towards Global Optimal Visual In-Context Learning Prompt Selection [50.174301123013045]
We propose a novel in-context example selection framework to identify the global optimal prompt.
Our method, dubbed Partial2Global, adopts a transformer-based list-wise ranker to provide a more comprehensive comparison.
The effectiveness of Partial2Global is validated through experiments on foreground segmentation, single object detection and image colorization.
arXiv Detail & Related papers (2024-05-24T07:07:24Z) - Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling [17.563853245956455]
In some settings, it is feasible to actively select which points to label.
We propose a natural algorithm based on fitting Gaussian mixtures for selecting which points to label.
The proposed algorithm outperforms state-of-the-art active learning methods when used with various meta-learning algorithms across several benchmark datasets.
arXiv Detail & Related papers (2023-11-06T05:18:01Z) - Exploiting Diversity of Unlabeled Data for Label-Efficient
Semi-Supervised Active Learning [57.436224561482966]
Active learning is a research area that addresses the issues of expensive labeling by selecting the most important samples for labeling.
We introduce a new diversity-based initial dataset selection algorithm to select the most informative set of samples for initial labeling in the active learning setting.
Also, we propose a novel active learning query strategy, which uses diversity-based sampling on consistency-based embeddings.
arXiv Detail & Related papers (2022-07-25T16:11:55Z) - Budget-aware Few-shot Learning via Graph Convolutional Network [56.41899553037247]
This paper tackles the problem of few-shot learning, which aims to learn new visual concepts from a few examples.
A common problem setting in few-shot classification assumes random sampling strategy in acquiring data labels.
We introduce a new budget-aware few-shot learning problem that aims to learn novel object categories.
arXiv Detail & Related papers (2022-01-07T02:46:35Z) - Multiple-criteria Based Active Learning with Fixed-size Determinantal
Point Processes [43.71112693633952]
We introduce a multiple-criteria based active learning algorithm, which incorporates three complementary criteria, i.e., informativeness, representativeness and diversity.
We show that our method performs significantly better and is more stable than other multiple-criteria based AL algorithms.
arXiv Detail & Related papers (2021-07-04T13:22:54Z) - One-Round Active Learning [13.25385227263705]
One-round active learning aims to select a subset of unlabeled data points that achieve the highest utility after being labeled.
We propose DULO, a general framework for one-round active learning based on the notion of data utility functions.
Our results demonstrate that while existing active learning approaches could succeed with multiple rounds, DULO consistently performs better in the one-round setting.
arXiv Detail & Related papers (2021-04-23T23:59:50Z) - Online Active Model Selection for Pre-trained Classifiers [72.84853880948894]
We design an online selective sampling approach that actively selects informative examples to label and outputs the best model with high probability at any round.
Our algorithm can be used for online prediction tasks for both adversarial and streams.
arXiv Detail & Related papers (2020-10-19T19:53:15Z) - A Comprehensive Benchmark Framework for Active Learning Methods in
Entity Matching [17.064993611446898]
In this paper, we build a unified active learning benchmark framework for EM.
The goal of the framework is to enable concrete guidelines for practitioners as to what active learning combinations will work well for EM.
Our framework also includes novel optimizations that improve the quality of the learned model by roughly 9% in terms of F1-score and reduce example selection latencies by up to 10x without affecting the quality of the model.
arXiv Detail & Related papers (2020-03-29T19:08:03Z) - Selecting Relevant Features from a Multi-domain Representation for
Few-shot Classification [91.67977602992657]
We propose a new strategy based on feature selection, which is both simpler and more effective than previous feature adaptation approaches.
We show that a simple non-parametric classifier built on top of such features produces high accuracy and generalizes to domains never seen during training.
arXiv Detail & Related papers (2020-03-20T15:44:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.