Bounded Memory Active Learning through Enriched Queries
- URL: http://arxiv.org/abs/2102.05047v1
- Date: Tue, 9 Feb 2021 19:00:00 GMT
- Title: Bounded Memory Active Learning through Enriched Queries
- Authors: Max Hopkins, Daniel Kane, Shachar Lovett, Michal Moshkovitz
- Abstract summary: Active learning is a paradigm in which data-hungry learning algorithms adaptively select informative examples in order to lower expensive labeling costs.
To combat this, a series of recent works have considered a model in which the learner may ask enriched queries beyond labels.
While such models have seen success in drastically lowering label costs, they tend to come at the expense of requiring large amounts of memory.
- Score: 28.116967200489192
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The explosive growth of easily-accessible unlabeled data has lead to growing
interest in active learning, a paradigm in which data-hungry learning
algorithms adaptively select informative examples in order to lower
prohibitively expensive labeling costs. Unfortunately, in standard worst-case
models of learning, the active setting often provides no improvement over
non-adaptive algorithms. To combat this, a series of recent works have
considered a model in which the learner may ask enriched queries beyond labels.
While such models have seen success in drastically lowering label costs, they
tend to come at the expense of requiring large amounts of memory. In this work,
we study what families of classifiers can be learned in bounded memory. To this
end, we introduce a novel streaming-variant of enriched-query active learning
along with a natural combinatorial parameter called lossless sample compression
that is sufficient for learning not only with bounded memory, but in a
query-optimal and computationally efficient manner as well. Finally, we give
three fundamental examples of classifier families with small, easy to compute
lossless compression schemes when given access to basic enriched queries:
axis-aligned rectangles, decision trees, and halfspaces in two dimensions.
Related papers
- Class incremental learning with probability dampening and cascaded gated classifier [4.285597067389559]
We propose a novel incremental regularisation approach called Margin Dampening and Cascaded Scaling.
The first combines a soft constraint and a knowledge distillation approach to preserve past knowledge while allowing forgetting new patterns.
We empirically show that our approach performs well on multiple benchmarks well-established baselines.
arXiv Detail & Related papers (2024-02-02T09:33:07Z) - Model Uncertainty based Active Learning on Tabular Data using Boosted
Trees [0.4667030429896303]
Supervised machine learning relies on the availability of good labelled data for model training.
Active learning is a sub-field of machine learning which helps in obtaining the labelled data efficiently.
arXiv Detail & Related papers (2023-10-30T14:29:53Z) - Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information.
We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting.
Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
arXiv Detail & Related papers (2023-09-15T17:10:51Z) - Ticketed Learning-Unlearning Schemes [57.89421552780526]
We propose a new ticketed model for learning--unlearning.
We provide space-efficient ticketed learning--unlearning schemes for a broad family of concept classes.
arXiv Detail & Related papers (2023-06-27T18:54:40Z) - Semiparametric Language Models Are Scalable Continual Learners [83.74414880208334]
Semiparametric language models (LMs) have shown promise in continuously learning from new text data.
We present a simple and intuitive approach called Selective Memorization (SeMem)
SeMem only memorizes difficult samples that the model is likely to struggle with.
arXiv Detail & Related papers (2023-03-02T17:15:02Z) - An Embarrassingly Simple Approach to Semi-Supervised Few-Shot Learning [58.59343434538218]
We propose a simple but quite effective approach to predict accurate negative pseudo-labels of unlabeled data from an indirect learning perspective.
Our approach can be implemented in just few lines of code by only using off-the-shelf operations.
arXiv Detail & Related papers (2022-09-28T02:11:34Z) - Efficient Active Learning with Abstention [12.315392649501101]
We develop the first computationally efficient active learning algorithm with abstention.
A key feature of the algorithm is that it avoids the undesirable "noise-seeking" behavior often seen in active learning.
arXiv Detail & Related papers (2022-03-31T18:34:57Z) - L2B: Learning to Bootstrap Robust Models for Combating Label Noise [52.02335367411447]
This paper introduces a simple and effective method, named Learning to Bootstrap (L2B)
It enables models to bootstrap themselves using their own predictions without being adversely affected by erroneous pseudo-labels.
It achieves this by dynamically adjusting the importance weight between real observed and generated labels, as well as between different samples through meta-learning.
arXiv Detail & Related papers (2022-02-09T05:57:08Z) - Machine Unlearning of Features and Labels [72.81914952849334]
We propose first scenarios for unlearning and labels in machine learning models.
Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
arXiv Detail & Related papers (2021-08-26T04:42:24Z) - Learning from Noisy Labels for Entity-Centric Information Extraction [17.50856935207308]
We propose a simple co-regularization framework for entity-centric information extraction.
These models are jointly optimized with task-specific loss, and are regularized to generate similar predictions.
In the end, we can take any of the trained models for inference.
arXiv Detail & Related papers (2021-04-17T22:49:12Z) - Fase-AL -- Adaptation of Fast Adaptive Stacking of Ensembles for
Supporting Active Learning [0.0]
This work presents the FASE-AL algorithm which induces classification models with non-labeled instances using Active Learning.
The algorithm achieves promising results in terms of the percentage of correctly classified instances.
arXiv Detail & Related papers (2020-01-30T17:25:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.