Search Strategies for Optimal Classification and Regression Trees

• URL: http://arxiv.org/abs/2607.28170v1
• Date: Thu, 30 Jul 2026 13:08:21 GMT
• Title: Search Strategies for Optimal Classification and Regression Trees
• Abstract summary: Optimal decision trees (ODTs) are machine learning models that globally optimize a given objective.<n>Recent work has proposed a variety of search strategies to improve scalability, but the precise contribution of each strategy remains unclear.<n>We introduce a general algorithmic framework for ODTs that instantiates previously used search strategies and enables the definition of new ones.
• Score: 6.627152091494143
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimal decision trees (ODTs) are compact, interpretable machine learning models that globally optimize a given objective, but their scalability remains challenging. While recent work has proposed a variety of search strategies to improve scalability, the precise contribution of each strategy remains unclear. To address this gap, we introduce a general algorithmic framework for ODTs that instantiates previously used search strategies and enables the definition of new ones. This provides a common lens through which to understand and compare different strategies, which we use to empirically investigate the effect of 18 search strategies. Compared to the state of the art, the best strategy in our evaluation achieves significantly better anytime performance for classification, and improves runtime by more than an order of magnitude for regression.

Related papers

• SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution [31.165726518569155]
  LLM-guided evolutionary search has emerged as a promising paradigm for automated algorithm discovery.<n>model is a modular strategy-space layer that elevates natural-language strategy descriptions to first-class population-level evolutionary state.<n>model improves the underlying evolutionary backbones in most settings, with particularly large gains on open-ended system optimization tasks.
  arXiv  Detail & Related papers  (2026-04-27T12:06:15Z)
• Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance [86.46794021499511]
  We show a previously underexplored gap between strategy usage and strategy executability.<n>We propose Selective Strategy Retrieval (SSR), a test-time framework that explicitly models executability.<n> SSR yields reliable and consistent improvements over direct solving, in-context learning, and single-source guidance.
  arXiv  Detail & Related papers  (2026-02-26T03:34:23Z)
• SAGE: Strategy-Adaptive Generation Engine for Query Rewriting [8.941793732446856]
  We introduce the Strategy-Adaptive Generation Engine (SAGE), which operationalizes expert-crafted strategies in an reinforcement learning framework.<n>SAGE achieves new state-of-the-art NDCG@10 results, but also uncovers a compelling emergent behavior.<n>Our findings demonstrate that strategy-guided RL, enhanced with nuanced reward shaping, offers a scalable, efficient, and more interpretable paradigm for developing the next generation of robust information retrieval systems.
  arXiv  Detail & Related papers  (2025-06-24T16:50:51Z)
• RL-finetuning LLMs from on- and off-policy data with a single algorithm [53.70731390624718]
  We introduce a novel reinforcement learning algorithm (AGRO) for fine-tuning large-language models.<n>AGRO leverages the concept of generation consistency, which states that the optimal policy satisfies the notion of consistency across any possible generation of the model.<n>We derive algorithms that find optimal solutions via the sample-based policy gradient and provide theoretical guarantees on their convergence.
  arXiv  Detail & Related papers  (2025-03-25T12:52:38Z)
• Stratify: Unifying Multi-Step Forecasting Strategies [0.0]
  Stratify is a framework that addresses multi-step forecasting and unifying existing strategies.<n>In over 84% of 1080 experiments, novel strategies in Stratify improved performance compared to all existing ones.<n>Our results are the most comprehensive benchmarking of known and novel forecasting strategies.
  arXiv  Detail & Related papers  (2024-12-29T16:06:46Z)
• Deep Reinforcement Learning for Online Optimal Execution Strategies [49.1574468325115]
  This paper tackles the challenge of learning non-Markovian optimal execution strategies in dynamic financial markets. We introduce a novel actor-critic algorithm based on Deep Deterministic Policy Gradient (DDPG) We show that our algorithm successfully approximates the optimal execution strategy.
  arXiv  Detail & Related papers  (2024-10-17T12:38:08Z)
• Towards a Unified View of Preference Learning for Large Language Models: A Survey [88.66719962576005]
  Large Language Models (LLMs) exhibit remarkably powerful capabilities. One of the crucial factors to achieve success is aligning the LLM's output with human preferences. We decompose all the strategies in preference learning into four components: model, data, feedback, and algorithm.
  arXiv  Detail & Related papers  (2024-09-04T15:11:55Z)
• Large-scale Fully-Unsupervised Re-Identification [78.47108158030213]
  We propose two strategies to learn from large-scale unlabeled data. The first strategy performs a local neighborhood sampling to reduce the dataset size in each without violating neighborhood relationships. A second strategy leverages a novel Re-Ranking technique, which has a lower time upper bound complexity and reduces the memory complexity from O(n2) to O(kn) with k n.
  arXiv  Detail & Related papers  (2023-07-26T16:19:19Z)
• Portfolio Search and Optimization for General Strategy Game-Playing [58.896302717975445]
  We propose a new algorithm for optimization and action-selection based on the Rolling Horizon Evolutionary Algorithm. For the optimization of the agents' parameters and portfolio sets we study the use of the N-tuple Bandit Evolutionary Algorithm. An analysis of the agents' performance shows that the proposed algorithm generalizes well to all game-modes and is able to outperform other portfolio methods.
  arXiv  Detail & Related papers  (2021-04-21T09:28:28Z)
• Can Global Optimization Strategy Outperform Myopic Strategy for Bayesian Parameter Estimation? [0.0]
  This paper provides a discouraging answer based on experimental simulations comparing the performance improvement and burden between global and myopic strategies. The added horizon in global strategies has negligible contributions to the improvement of optimal global utility other than the most immediate next steps.
  arXiv  Detail & Related papers  (2020-07-01T10:31:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.