From Weakly Supervised Learning to Active Learning
- URL: http://arxiv.org/abs/2209.11629v1
- Date: Fri, 23 Sep 2022 14:55:43 GMT
- Title: From Weakly Supervised Learning to Active Learning
- Authors: Vivien Cabannes
- Abstract summary: This thesis is motivated by the question: can we derive a more generic framework than the one of supervised learning?
We model weak supervision as giving, rather than a unique target, a set of target candidates.
We argue that one should look for an optimistic'' function that matches most of the observations. This allows us to derive a principle to disambiguate partial labels.
- Score: 1.52292571922932
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Applied mathematics and machine computations have raised a lot of hope since
the recent success of supervised learning. Many practitioners in industries
have been trying to switch from their old paradigms to machine learning.
Interestingly, those data scientists spend more time scrapping, annotating and
cleaning data than fine-tuning models. This thesis is motivated by the
following question: can we derive a more generic framework than the one of
supervised learning in order to learn from clutter data?
This question is approached through the lens of weakly supervised learning,
assuming that the bottleneck of data collection lies in annotation. We model
weak supervision as giving, rather than a unique target, a set of target
candidates. We argue that one should look for an ``optimistic'' function that
matches most of the observations. This allows us to derive a principle to
disambiguate partial labels. We also discuss the advantage to incorporate
unsupervised learning techniques into our framework, in particular manifold
regularization approached through diffusion techniques, for which we derived a
new algorithm that scales better with input dimension then the baseline method.
Finally, we switch from passive to active weakly supervised learning,
introducing the ``active labeling'' framework, in which a practitioner can
query weak information about chosen data. Among others, we leverage the fact
that one does not need full information to access stochastic gradients and
perform stochastic gradient descent.
Related papers
- Attribute-to-Delete: Machine Unlearning via Datamodel Matching [65.13151619119782]
Machine unlearning -- efficiently removing a small "forget set" training data on a pre-divertrained machine learning model -- has recently attracted interest.
Recent research shows that machine unlearning techniques do not hold up in such a challenging setting.
arXiv Detail & Related papers (2024-10-30T17:20:10Z) - Enhancing Consistency and Mitigating Bias: A Data Replay Approach for
Incremental Learning [100.7407460674153]
Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks.
To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks.
However, it is not expected in practice considering the memory constraint or data privacy issue.
As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
arXiv Detail & Related papers (2024-01-12T12:51:12Z) - Zero-shot Active Learning Using Self Supervised Learning [11.28415437676582]
We propose a new Active Learning approach which is model agnostic as well as one doesn't require an iterative process.
We aim to leverage self-supervised learnt features for the task of Active Learning.
arXiv Detail & Related papers (2024-01-03T11:49:07Z) - Robust Machine Learning by Transforming and Augmenting Imperfect
Training Data [6.928276018602774]
This thesis explores several data sensitivities of modern machine learning.
We first discuss how to prevent ML from codifying prior human discrimination measured in the training data.
We then discuss the problem of learning from data containing spurious features, which provide predictive fidelity during training but are unreliable upon deployment.
arXiv Detail & Related papers (2023-12-19T20:49:28Z) - One-bit Supervision for Image Classification: Problem, Solution, and
Beyond [114.95815360508395]
This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification.
We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm.
In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
arXiv Detail & Related papers (2023-11-26T07:39:00Z) - Towards Label-Efficient Incremental Learning: A Survey [42.603603392991715]
We study incremental learning, where a learner is required to adapt to an incoming stream of data with a varying distribution.
We identify three subdivisions, namely semi-, few-shot- and self-supervised learning to reduce labeling efforts.
arXiv Detail & Related papers (2023-02-01T10:24:55Z) - Adversarial Auto-Augment with Label Preservation: A Representation
Learning Principle Guided Approach [95.74102207187545]
We show that a prior-free autonomous data augmentation's objective can be derived from a representation learning principle.
We then propose a practical surrogate to the objective that can be efficiently optimized and integrated seamlessly into existing methods.
arXiv Detail & Related papers (2022-11-02T02:02:51Z) - Learning to Predict Gradients for Semi-Supervised Continual Learning [36.715712711431856]
Key challenge for machine intelligence is to learn new visual concepts without forgetting the previously acquired knowledge.
There is a gap between existing supervised continual learning and human-like intelligence, where human is able to learn from both labeled and unlabeled data.
We formulate a new semi-supervised continual learning method, which can be generically applied to existing continual learning models.
arXiv Detail & Related papers (2022-01-23T06:45:47Z) - Reasoning-Modulated Representations [85.08205744191078]
We study a common setting where our task is not purely opaque.
Our approach paves the way for a new class of data-efficient representation learning.
arXiv Detail & Related papers (2021-07-19T13:57:13Z) - Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
We introduce a fast optimization-based meta-learning method for few-shot classification.
Our strategy enables important aspects of the base learner objective to be learned during meta-training.
We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
arXiv Detail & Related papers (2020-10-01T15:59:31Z) - A Survey on Self-supervised Pre-training for Sequential Transfer
Learning in Neural Networks [1.1802674324027231]
Self-supervised pre-training for transfer learning is becoming an increasingly popular technique to improve state-of-the-art results using unlabeled data.
We provide an overview of the taxonomy for self-supervised learning and transfer learning, and highlight some prominent methods for designing pre-training tasks across different domains.
arXiv Detail & Related papers (2020-07-01T22:55:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.