Deep Active Learning in the Open World

• URL: http://arxiv.org/abs/2411.06353v1
• Date: Sun, 10 Nov 2024 04:04:20 GMT
• Title: Deep Active Learning in the Open World
• Authors: Tian Xie, Jifan Zhang, Haoyue Bai, Robert Nowak,
• Abstract summary: Machine learning models deployed in open-world scenarios often encounter unfamiliar conditions and perform poorly in unanticipated situations. We introduce ALOE, a novel active learning algorithm for open-world environments designed to enhance model adaptation by incorporating new OOD classes. Our findings reveal a crucial tradeoff between enhancing known-class performance and discovering new classes, setting the stage for future advancements in open-world machine learning.
• Score: 13.2318584850986
• License:
• Abstract: Machine learning models deployed in open-world scenarios often encounter unfamiliar conditions and perform poorly in unanticipated situations. As AI systems advance and find application in safety-critical domains, effectively handling out-of-distribution (OOD) data is crucial to building open-world learning systems. In this work, we introduce ALOE, a novel active learning algorithm for open-world environments designed to enhance model adaptation by incorporating new OOD classes via a two-stage approach. First, diversity sampling selects a representative set of examples, followed by energy-based OOD detection to prioritize likely unknown classes for annotation. This strategy accelerates class discovery and learning, even under constrained annotation budgets. Evaluations on three long-tailed image classification benchmarks demonstrate that ALOE outperforms traditional active learning baselines, effectively expanding known categories while balancing annotation cost. Our findings reveal a crucial tradeoff between enhancing known-class performance and discovering new classes, setting the stage for future advancements in open-world machine learning.

Related papers

• A Unified and General Framework for Continual Learning [58.72671755989431]
  Continual Learning (CL) focuses on learning from dynamic and changing data distributions while retaining previously acquired knowledge. Various methods have been developed to address the challenge of catastrophic forgetting, including regularization-based, Bayesian-based, and memory-replay-based techniques. This research aims to bridge this gap by introducing a comprehensive and overarching framework that encompasses and reconciles these existing methodologies.
  arXiv  Detail & Related papers  (2024-03-20T02:21:44Z)
• Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theory [15.875140867859209]
  This thesis makes contributions to the realm of machine learning, specifically in the context of open-world scenarios. Research investigates two intertwined steps essential for open-world machine learning: Out-of-distribution (OOD) Detection and Open-world Representation Learning (ORL)
  arXiv  Detail & Related papers  (2023-10-10T00:25:21Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [71.63186089279218]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• SegPrompt: Boosting Open-world Segmentation via Category-level Prompt Learning [49.17344010035996]
  Open-world instance segmentation (OWIS) models detect unknown objects in a class-agnostic manner. Previous OWIS approaches completely erase category information during training to keep the model's ability to generalize to unknown objects. We propose a novel training mechanism termed SegPrompt that uses category information to improve the model's class-agnostic segmentation ability.
  arXiv  Detail & Related papers  (2023-08-12T11:25:39Z)
• Open Long-Tailed Recognition in a Dynamic World [82.91025831618545]
  Real world data often exhibits a long-tailed and open-ended (with unseen classes) distribution. A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes) We define Open Long-Tailed Recognition++ as learning from such naturally distributed data and optimizing for the classification accuracy over a balanced test set.
  arXiv  Detail & Related papers  (2022-08-17T15:22:20Z)
• SHELS: Exclusive Feature Sets for Novelty Detection and Continual Learning Without Class Boundaries [22.04165296584446]
  We introduce a Sparse High-level-Exclusive, Low-level-Shared feature representation (SHELS) SHELS encourages learning exclusive sets of high-level features and essential, shared low-level features. We show that using SHELS for novelty detection results in statistically significant improvements over state-of-the-art OOD detection approaches.
  arXiv  Detail & Related papers  (2022-06-28T03:09:55Z)
• Active Learning for Open-set Annotation [38.739845944840454]
  We propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-01-18T06:11:51Z)
• Bayesian Embeddings for Few-Shot Open World Recognition [60.39866770427436]
  We extend embedding-based few-shot learning algorithms to the open-world recognition setting. We benchmark our framework on open-world extensions of the common MiniImageNet and TieredImageNet few-shot learning datasets.
  arXiv  Detail & Related papers  (2021-07-29T00:38:47Z)
• Open-World Semi-Supervised Learning [66.90703597468377]
  We introduce a new open-world semi-supervised learning setting in which the model is required to recognize previously seen classes. We propose ORCA, an approach that learns to simultaneously classify and cluster the data. We demonstrate that ORCA accurately discovers novel classes and assigns samples to previously seen classes on benchmark image classification datasets.
  arXiv  Detail & Related papers  (2021-02-06T07:11:07Z)
• Task-Adaptive Negative Class Envision for Few-Shot Open-Set Recognition [36.53830822788852]
  We study the problem of few-shot open-set recognition (FSOR), which learns a recognition system robust to queries from new sources. We propose a novel task-adaptive negative class envision method (TANE) to model the open world. Our approach significantly improves the state-of-the-art performance on few-shot open-set recognition.
  arXiv  Detail & Related papers  (2020-12-24T02:30:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.