Cost-aware Stopping for Bayesian Optimization

• URL: http://arxiv.org/abs/2507.12453v1
• Date: Wed, 16 Jul 2025 17:54:14 GMT
• Title: Cost-aware Stopping for Bayesian Optimization
• Authors: Qian Xie, Linda Cai, Alexander Terenin, Peter I. Frazier, Ziv Scully,
• Abstract summary: We propose a cost-aware stopping rule for Bayesian optimization that adapts to varying evaluation costs and is free of tuning.<n>We prove a theoretical guarantee bounding the expected cumulative evaluation cost incurred by our stopping rule when paired with state-of-the-art acquisition functions.
• Score: 53.34052774820105
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In automated machine learning, scientific discovery, and other applications of Bayesian optimization, deciding when to stop evaluating expensive black-box functions is an important practical consideration. While several adaptive stopping rules have been proposed, in the cost-aware setting they lack guarantees ensuring they stop before incurring excessive function evaluation costs. We propose a cost-aware stopping rule for Bayesian optimization that adapts to varying evaluation costs and is free of heuristic tuning. Our rule is grounded in a theoretical connection to state-of-the-art cost-aware acquisition functions, namely the Pandora's Box Gittins Index (PBGI) and log expected improvement per cost. We prove a theoretical guarantee bounding the expected cumulative evaluation cost incurred by our stopping rule when paired with these two acquisition functions. In experiments on synthetic and empirical tasks, including hyperparameter optimization and neural architecture size search, we show that combining our stopping rule with the PBGI acquisition function consistently matches or outperforms other acquisition-function--stopping-rule pairs in terms of cost-adjusted simple regret, a metric capturing trade-offs between solution quality and cumulative evaluation cost.

Related papers

• Cost-aware Bayesian Optimization via the Pandora's Box Gittins Index [57.045952766988925]
  We develop a previously-unexplored connection between cost-aware Bayesian optimization and the Pandora's Box problem, a decision problem from economics.<n>Our work constitutes a first step towards integrating techniques from Gittins index theory into Bayesian optimization.
  arXiv  Detail & Related papers  (2024-06-28T17:20:13Z)
• Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation [55.75188191403343]
  We introduce utility, which is a function predefined by each user and describes the trade-off between cost and performance of BO. We validate our algorithm on various LC datasets and found it outperform all the previous multi-fidelity BO and transfer-BO baselines we consider.
  arXiv  Detail & Related papers  (2024-05-28T07:38:39Z)
• Generalizing Bayesian Optimization with Decision-theoretic Entropies [102.82152945324381]
  We consider a generalization of Shannon entropy from work in statistical decision theory. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures. We then show how alternative choices for the loss yield a flexible family of acquisition functions.
  arXiv  Detail & Related papers  (2022-10-04T04:43:58Z)
• Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs [28.254408148839644]
  We propose a non-myopic acquisition function that generalizes classical expected improvement to the setting of heterogeneous evaluation costs. Our acquisition function outperforms existing methods in a variety of synthetic and real problems.
  arXiv  Detail & Related papers  (2021-11-12T02:18:26Z)
• Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions [74.00030431081751]
  We formalize the notion of user-specific cost functions and introduce a new method for identifying actionable recourses for users. Our method satisfies up to 25.89 percentage points more users compared to strong baseline methods.
  arXiv  Detail & Related papers  (2021-11-01T19:49:35Z)
• A Nonmyopic Approach to Cost-Constrained Bayesian Optimization [10.078368988372247]
  We formulate cost-constrained BO as a constrained Markov decision process (CMDP) We develop an efficient rollout approximation to the optimal CMDP policy that takes both the cost and future iterations into account.
  arXiv  Detail & Related papers  (2021-06-10T22:44:37Z)
• Cost-Efficient Online Hyperparameter Optimization [94.60924644778558]
  We propose an online HPO algorithm that reaches human expert-level performance within a single run of the experiment. Our proposed online HPO algorithm reaches human expert-level performance within a single run of the experiment, while incurring only modest computational overhead compared to regular training.
  arXiv  Detail & Related papers  (2021-01-17T04:55:30Z)
• Pareto-efficient Acquisition Functions for Cost-Aware Bayesian Optimization [5.459427541271035]
  We show how to make cost-aware Bayesian optimization for black-box functions. On 144 real-world black-box function optimization problems, our solution brings up to 50% speed-ups. We also revisit the common choice of Gaussian process cost models, showing that simple, low-variance cost models predict training times effectively.
  arXiv  Detail & Related papers  (2020-11-23T15:06:07Z)
• Cost-aware Bayesian Optimization [6.75013674088437]
  Cost-aware BO measures convergence with alternative cost metrics such as time, energy, or money. We introduce Cost Apportioned BO (CArBO), which attempts to minimize an objective function in as little cost as possible.
  arXiv  Detail & Related papers  (2020-03-22T14:51:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.