Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimization

• URL: http://arxiv.org/abs/2606.04866v1
• Date: Wed, 03 Jun 2026 13:32:17 GMT
• Title: Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimization
• Abstract summary: We provide the first distribution-dependent sample complexity bounds for multi-fidelity HPO with priors.<n>We show that informative priors concentrate probability mass on near-optimal arms, yield reductions in the number of required evaluations, whereas baseline performance is recovered with uninformative or misleading priors.
• Score: 14.49729393385318
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large-scale hyperparameter optimization (HPO) in automated machine learning (AutoML) consumes substantial computational resources, raising growing concerns about scalability and energy efficiency. Existing methods use prior information heuristically to accelerate both black-box and multi-fidelity settings, but they lack a characterization of how prior informativeness quantitatively reduces sample complexity. In this work, we provide the first distribution-dependent sample complexity bounds for multi-fidelity HPO with priors through the formal lens of fixed-budget best-arm identification. By modeling priors directly over arm means as configuration performance, we derive explicit, distribution-dependent error bounds that quantify the relationship between priors and evaluation budget. Our analysis shows that informative priors, which concentrate probability mass on near-optimal arms, yield reductions in the number of required evaluations, whereas baseline performance is recovered with uninformative or misleading priors. We conduct proof-of-concept experiments on a synthetic benchmark and on LCBench, a common multi-fidelity HPO benchmark for deep learning, to confirm our theoretical results, achieving up to 90% budget reduction while retaining solution quality. Together, our results provide a principled foundation for prior-guided and compute-efficient green AutoML.

Related papers

• A Multi-Agent System for IPMSM Design Optimization via an FEA-AI Hybrid Approach [4.798313004706817]
  Interior permanent magnet synchronous motor (IPMSM) design requires balancing conflicting objectives and multi-physics constraints.<n>We propose an end-to-end automated IPMSM design optimization framework that integrates retrieval-augmented generation (RAG) for structured problem definition.
  arXiv  Detail & Related papers  (2026-06-08T05:07:46Z)
• The Value of Mechanistic Priors in Sequential Decision Making [41.418304743101174]
  Hybrid mechanistic models, physical priors with learned residuals, promise to reduce the data required for good decisions, but have no computable criterion to test this.<n>We characterize the value of mechanistic priors in sequential decision-making within both and burn-in regimes.
  arXiv  Detail & Related papers  (2026-05-11T05:43:53Z)
• ADKO: Agentic Decentralized Knowledge Optimization [21.537388090124196]
  Agentic Decentralized Knowledge Optimization (ADKO) is a framework for collaborative black-box optimization across autonomous agents.<n>Each agent maintains a private Gaussian Process (GP) surrogate trained on local data and communicates only through knowledge tokens.<n>We provide the first formal analysis of dual information loss: token compression, quantified via mutual-information-based fidelity, and LM approximation, decomposed into bias and noise.
  arXiv  Detail & Related papers  (2026-05-08T15:23:53Z)
• Unbiased Dynamic Pruning for Efficient Group-Based Policy Optimization [60.87651283510059]
  Group Relative Policy Optimization (GRPO) effectively scales LLM reasoning but incurs prohibitive computational costs.<n>We propose Dynamic Pruning Policy Optimization (DPPO), a framework that enables dynamic pruning while preserving unbiased gradient estimation.<n>To mitigate the data sparsity induced by pruning, we introduce Dense Prompt Packing, a window-based greedy strategy.
  arXiv  Detail & Related papers  (2026-03-04T14:48:53Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• Calibrating Agent-Based Financial Markets Simulators with Pretrainable Automatic Posterior Transformation-Based Surrogates [5.002657036975061]
  Calibrating Agent-Based Models (ABMs) is an important optimization problem for simulating the complex social systems.<n>The goal is to identify the optimal parameter of a given ABM by minimizing the discrepancy between the simulated data and the real-world observations.<n>Existing methods face two key limitations: 1) surrogating the original evaluation function is hard due the nonlinear yet multi-modal nature of the ABMs, and 2) the commonly used surrogates cannot share the optimization experience among multiple calibration tasks.<n>This work proposes Automatic posterior transformation with Negatively Correlated Search and Adaptive Trust-Region.
  arXiv  Detail & Related papers  (2026-01-11T14:05:26Z)
• Repurposing Synthetic Data for Fine-grained Search Agent Supervision [81.95597592711688]
  LLM-based search agents are increasingly trained on entity-centric synthetic data.<n> prevailing training methods discard this rich entity information, relying instead on sparse, outcome-based rewards.<n>We introduce Entity-aware Group Relative Policy Optimization (E-GRPO), a novel framework that formulates a dense entity-aware reward function.
  arXiv  Detail & Related papers  (2025-10-28T17:50:40Z)
• EconProver: Towards More Economical Test-Time Scaling for Automated Theorem Proving [64.15371139980802]
  Large Language Models (LLMs) have recently advanced the field of Automated Theorem Proving (ATP)<n>We show that different test-time scaling strategies for ATP models introduce significant computational overhead for inference.<n>We propose two complementary methods that can be integrated into a unified EconRL pipeline for amplified benefits.
  arXiv  Detail & Related papers  (2025-09-16T03:00:13Z)
• Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysis [17.98292973608615]
  We propose a novel Uncertainty-Aware Collaborative System (U-ACS) that orchestrates a powerful MLLM and a lightweight baseline model for multimodal sentiment analysis.<n>Our proposed method achieves state-of-the-art performance, while requiring only a fraction of the computational resources compared to using a standalone MLLM.
  arXiv  Detail & Related papers  (2025-08-27T16:01:58Z)
• Preference Optimization for Combinatorial Optimization Problems [54.87466279363487]
  Reinforcement Learning (RL) has emerged as a powerful tool for neural optimization, enabling models learns that solve complex problems without requiring expert knowledge.<n>Despite significant progress, existing RL approaches face challenges such as diminishing reward signals and inefficient exploration in vast action spaces.<n>We propose Preference Optimization, a novel method that transforms quantitative reward signals into qualitative preference signals via statistical comparison modeling.
  arXiv  Detail & Related papers  (2025-05-13T16:47:00Z)
• Supervised Optimism Correction: Be Confident When LLMs Are Sure [91.7459076316849]
  We establish a novel theoretical connection between supervised fine-tuning and offline reinforcement learning.<n>We show that the widely used beam search method suffers from unacceptable over-optimism.<n>We propose Supervised Optimism Correction, which introduces a simple yet effective auxiliary loss for token-level $Q$-value estimations.
  arXiv  Detail & Related papers  (2025-04-10T07:50:03Z)
• "Over-optimizing" for Normality: Budget-constrained Uncertainty Quantification for Contextual Decision-making [1.2997358113120427]
  We study uncertainty quantification for contextual optimization using weighted sample average approximation (wSAA)<n>We establish central limit theorems for wSAA and construct algorithm-normality-based confidence intervals for optimal conditional expected costs.<n>We show that over-optimizing'' (running more iterations than the nominal rule) mitigates this misspecification and preserves normality, at the expense of a slight slowdown in the convergence rate of the budget-constrained estimator.
  arXiv  Detail & Related papers  (2025-03-17T02:31:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.