Active Learning for Open-set Annotation

• URL: http://arxiv.org/abs/2201.06758v1
• Date: Tue, 18 Jan 2022 06:11:51 GMT
• Title: Active Learning for Open-set Annotation
• Authors: Kun-Peng Ning, Xun Zhao, Yu Li, Sheng-Jun Huang
• Abstract summary: We propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods.
• Score: 38.739845944840454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing active learning studies typically work in the closed-set setting by assuming that all data examples to be labeled are drawn from known classes. However, in real annotation tasks, the unlabeled data usually contains a large amount of examples from unknown classes, resulting in the failure of most active learning methods. To tackle this open-set annotation (OSA) problem, we propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The LfOSA framework introduces an auxiliary network to model the per-example max activation value (MAV) distribution with a Gaussian Mixture Model, which can dynamically select the examples with highest probability from known classes in the unlabeled set. Moreover, by reducing the temperature $T$ of the loss function, the detection model will be further optimized by exploiting both known and unknown supervision. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods. To the best of our knowledge, this is the first work of active learning for open-set annotation.

Related papers

• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes. Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes. We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• MALADY: Multiclass Active Learning with Auction Dynamics on Graphs [0.9831489366502301]
  We introduce the Multiclass Active Learning with Auction Dynamics on Graphs (MALADY) framework for efficient active learning. We generalize the auction dynamics algorithm on similarity graphs for semi-supervised learning in [24] to incorporate a more general optimization functional. We also introduce a novel active learning acquisition function that uses the dual variable of the auction algorithm to measure the uncertainty in the classifier to prioritize queries near the decision boundaries between different classes.
  arXiv  Detail & Related papers  (2024-09-14T16:20:26Z)
• Robust Semi-supervised Learning by Wisely Leveraging Open-set Data [48.67897991121204]
  Open-set Semi-supervised Learning (OSSL) holds a realistic setting that unlabeled data may come from classes unseen in the labeled set. We propose Wise Open-set Semi-supervised Learning (WiseOpen), a generic OSSL framework that selectively leverages the open-set data for training the model.
  arXiv  Detail & Related papers  (2024-05-11T10:22:32Z)
• Inconsistency-Based Data-Centric Active Open-Set Annotation [6.652785290214744]
  NEAT is a data-centric active learning method that actively annotates open-set data. NEAT achieves significantly better performance than state-of-the-art active learning methods for active open-set annotation.
  arXiv  Detail & Related papers  (2024-01-10T04:18:02Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Active Learning Principles for In-Context Learning with Large Language Models [65.09970281795769]
  This paper investigates how Active Learning algorithms can serve as effective demonstration selection methods for in-context learning. We show that in-context example selection through AL prioritizes high-quality examples that exhibit low uncertainty and bear similarity to the test examples.
  arXiv  Detail & Related papers  (2023-05-23T17:16:04Z)
• Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition [47.69171542776917]
  We find that features with richer semantic diversity can significantly improve the open-set performance under the same uncertainty scores. A novel Prototypical Similarity Learning (PSL) framework is proposed to keep the instance variance within the same class to retain more IS information.
  arXiv  Detail & Related papers  (2023-03-25T04:07:36Z)
• Optimizing Active Learning for Low Annotation Budgets [6.753808772846254]
  In deep learning, active learning is usually implemented as an iterative process in which successive deep models are updated via fine tuning. We tackle this issue by using an approach inspired by transfer learning. We introduce a novel acquisition function which exploits the iterative nature of AL process to select samples in a more robust fashion.
  arXiv  Detail & Related papers  (2022-01-18T18:53:10Z)
• MCDAL: Maximum Classifier Discrepancy for Active Learning [74.73133545019877]
  Recent state-of-the-art active learning methods have mostly leveraged Generative Adversarial Networks (GAN) for sample acquisition. We propose in this paper a novel active learning framework that we call Maximum Discrepancy for Active Learning (MCDAL) In particular, we utilize two auxiliary classification layers that learn tighter decision boundaries by maximizing the discrepancies among them.
  arXiv  Detail & Related papers  (2021-07-23T06:57:08Z)
• Semi-supervised Active Learning for Instance Segmentation via Scoring Predictions [25.408505612498423]
  We propose a novel and principled semi-supervised active learning framework for instance segmentation. Specifically, we present an uncertainty sampling strategy named Triplet Scoring Predictions (TSP) to explicitly incorporate samples ranking clues from classes, bounding boxes and masks. Results on medical images datasets demonstrate that the proposed method results in the embodiment of knowledge from available data in a meaningful way.
  arXiv  Detail & Related papers  (2020-12-09T02:36:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.