BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning

• URL: http://arxiv.org/abs/2603.13109v1
• Date: Fri, 13 Mar 2026 16:05:37 GMT
• Title: BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning
• Abstract summary: Best-of-Strategy Selector (BoSS) is a scalable oracle strategy designed for large-scale active learning scenarios.<n>BoSS constructs a set of candidate batches through an ensemble of selection strategies and then selects the batch yielding the highest performance gain.<n>As an ensemble of selection strategies, BoSS can be easily extended with new state-of-the-art strategies as they emerge.
• Score: 6.847756045870736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Active learning (AL) aims to reduce annotation costs while maximizing model performance by iteratively selecting valuable instances. While foundation models have made it easier to identify these instances, existing selection strategies still lack robustness across different models, annotation budgets, and datasets. To highlight the potential weaknesses of existing AL strategies and provide a reference point for research, we explore oracle strategies, i.e., strategies that approximate the optimal selection by accessing ground-truth information unavailable in practical AL scenarios. Current oracle strategies, however, fail to scale effectively to large datasets and complex deep neural networks. To tackle these limitations, we introduce the Best-of-Strategy Selector (BoSS), a scalable oracle strategy designed for large-scale AL scenarios. BoSS constructs a set of candidate batches through an ensemble of selection strategies and then selects the batch yielding the highest performance gain. As an ensemble of selection strategies, BoSS can be easily extended with new state-of-the-art strategies as they emerge, ensuring it remains a reliable oracle strategy in the future. Our evaluation demonstrates that i) BoSS outperforms existing oracle strategies, ii) state-of-the-art AL strategies still fall noticeably short of oracle performance, especially in large-scale datasets with many classes, and iii) one possible solution to counteract the inconsistent performance of AL strategies might be to employ an ensemble-based approach for the selection.

Related papers

• Search Strategies for Optimal Classification and Regression Trees [6.627152091494143]
  Optimal decision trees (ODTs) are machine learning models that globally optimize a given objective.<n>Recent work has proposed a variety of search strategies to improve scalability, but the precise contribution of each strategy remains unclear.<n>We introduce a general algorithmic framework for ODTs that instantiates previously used search strategies and enables the definition of new ones.
  arXiv  Detail & Related papers  (2026-07-30T13:08:21Z)
• StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction [76.50439715778869]
  StraTA is a framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning.<n>Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance.
  arXiv  Detail & Related papers  (2026-05-07T17:51:16Z)
• Automating aggregation strategy selection in federated learning [0.0]
  We present an end-to-end framework that automates, streamlines, and adapts aggregation strategy selection for federated learning.<n>Our approach enhances robustness and generalisation under non-IID conditions while reducing the need for manual intervention.
  arXiv  Detail & Related papers  (2026-04-09T10:08:28Z)
• Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance [86.46794021499511]
  We show a previously underexplored gap between strategy usage and strategy executability.<n>We propose Selective Strategy Retrieval (SSR), a test-time framework that explicitly models executability.<n> SSR yields reliable and consistent improvements over direct solving, in-context learning, and single-source guidance.
  arXiv  Detail & Related papers  (2026-02-26T03:34:23Z)
• Reinforced Strategy Optimization for Conversational Recommender Systems via Network-of-Experts [63.412646471177645]
  We propose a novel Reinforced Strategy Optimization (RSO) method for Conversational Recommender Systems (CRSs)<n>RSO decomposes the process of generating strategy-driven response decisions into the macro-level strategy planning and micro-level strategy adaptation.<n>Experiments show that RSO significantly improves interaction performance compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-09-30T11:12:01Z)
• Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs [49.995906301946]
  Existing methods usually leverage a fixed strategy to guide Large Language Models (LLMs) to perform mathematical reasoning.<n>Our analysis reveals that the single strategy cannot adapt to problem-specific requirements and thus overlooks the trade-off between effectiveness and efficiency.<n>We propose Planning and Routing through Instance-Specific Modeling (PRISM), a novel framework that decouples mathematical reasoning into two stages: strategy planning and targeted execution.
  arXiv  Detail & Related papers  (2025-09-29T07:22:41Z)
• EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning [69.55982246413046]
  We propose explicit policy optimization (EPO) for strategic reasoning.<n>We train the strategic reasoning model via multi-turn reinforcement learning (RL),utilizing process rewards and iterative self-play.<n>Our findings reveal various collaborative reasoning mechanisms emergent in EPO and its effectiveness in generating novel strategies.
  arXiv  Detail & Related papers  (2025-02-18T03:15:55Z)
• Stratify: Unifying Multi-Step Forecasting Strategies [0.0]
  Stratify is a framework that addresses multi-step forecasting and unifying existing strategies.<n>In over 84% of 1080 experiments, novel strategies in Stratify improved performance compared to all existing ones.<n>Our results are the most comprehensive benchmarking of known and novel forecasting strategies.
  arXiv  Detail & Related papers  (2024-12-29T16:06:46Z)
• Scalable and Equitable Math Problem Solving Strategy Prediction in Big Educational Data [2.86829428083307]
  We develop an embedding called MVec where we learn a representation based on the mastery of students. We then cluster these embeddings with a non-parametric clustering method. We show that our approach can scale up to achieve high accuracy by training on a small sample of a large dataset.
  arXiv  Detail & Related papers  (2023-08-07T19:51:10Z)
• ALE: A Simulation-Based Active Learning Evaluation Framework for the Parameter-Driven Comparison of Query Strategies for NLP [3.024761040393842]
  Active Learning (AL) proposes promising data points to annotators they annotate next instead of a subsequent or random sample. This method is supposed to save annotation effort while maintaining model performance. We introduce a reproducible active learning evaluation framework for the comparative evaluation of AL strategies in NLP.
  arXiv  Detail & Related papers  (2023-08-01T10:42:11Z)
• ImitAL: Learned Active Learning Strategy on Synthetic Data [30.595138995552748]
  We propose ImitAL, a domain-independent novel query strategy, which encodes AL as a learning-to-rank problem. We train ImitAL on large-scale simulated AL runs on purely synthetic datasets. To show that ImitAL was successfully trained, we perform an extensive evaluation comparing our strategy on 13 different datasets.
  arXiv  Detail & Related papers  (2022-08-24T16:17:53Z)
• SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning [58.26384597768118]
  SemiNLL is a versatile framework that combines SS strategies and SSL models in an end-to-end manner. Our framework can absorb various SS strategies and SSL backbones, utilizing their power to achieve promising performance.
  arXiv  Detail & Related papers  (2020-12-02T01:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.