An adaptive approach to Bayesian Optimization with switching costs

• URL: http://arxiv.org/abs/2405.08973v1
• Date: Tue, 14 May 2024 21:55:02 GMT
• Title: An adaptive approach to Bayesian Optimization with switching costs
• Authors: Stefan Pricopie, Richard Allmendinger, Manuel Lopez-Ibanez, Clyde Fare, Matt Benatan, Joshua Knowles,
• Abstract summary: We investigate modifications to Bayesian Optimization for a resource-constrained setting of sequential experimental design. We adapt two process-constrained batch algorithms to this sequential problem formulation, and propose two new methods: one cost-aware and one cost-ignorant.
• Score: 0.8246494848934447
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate modifications to Bayesian Optimization for a resource-constrained setting of sequential experimental design where changes to certain design variables of the search space incur a switching cost. This models the scenario where there is a trade-off between evaluating more while maintaining the same setup, or switching and restricting the number of possible evaluations due to the incurred cost. We adapt two process-constrained batch algorithms to this sequential problem formulation, and propose two new methods: one cost-aware and one cost-ignorant. We validate and compare the algorithms using a set of 7 scalable test functions in different dimensionalities and switching-cost settings for 30 total configurations. Our proposed cost-aware hyperparameter-free algorithm yields comparable results to tuned process-constrained algorithms in all settings we considered, suggesting some degree of robustness to varying landscape features and cost trade-offs. This method starts to outperform the other algorithms with increasing switching-cost. Our work broadens out from other recent Bayesian Optimization studies in resource-constrained settings that consider a batch setting only. While the contributions of this work are relevant to the general class of resource-constrained problems, they are particularly relevant to problems where adaptability to varying resource availability is of high importance

Related papers

• Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization [52.80408805368928]
  We introduce a novel greedy-style subset selection algorithm for batch acquisition. Our experiments on the red fluorescent proteins show that our proposed method achieves the baseline performance in 1.69x fewer queries.
  arXiv  Detail & Related papers  (2024-06-21T05:57:08Z)
• Efficient Controllable Multi-Task Architectures [85.76598445904374]
  We propose a multi-task model consisting of a shared encoder and task-specific decoders where both encoder and decoder channel widths are slimmable. Our key idea is to control the task importance by varying the capacities of task-specific decoders, while controlling the total computational cost. This improves overall accuracy by allowing a stronger encoder for a given budget, increases control over computational cost, and delivers high-quality slimmed sub-architectures.
  arXiv  Detail & Related papers  (2023-08-22T19:09:56Z)
• Movement Penalized Bayesian Optimization with Application to Wind Energy Systems [84.7485307269572]
  Contextual Bayesian optimization (CBO) is a powerful framework for sequential decision-making given side information. In this setting, the learner receives context (e.g., weather conditions) at each round, and has to choose an action (e.g., turbine parameters) Standard algorithms assume no cost for switching their decisions at every round, but in many practical applications, there is a cost associated with such changes, which should be minimized.
  arXiv  Detail & Related papers  (2022-10-14T20:19:32Z)
• Optimal Parameter-free Online Learning with Switching Cost [47.415099037249085]
  -freeness in online learning refers to the adaptivity of an algorithm with respect to the optimal decision in hindsight. In this paper, we design such algorithms in the presence of switching cost - the latter penalizes the optimistic updates required by parameter-freeness. We propose a simple yet powerful algorithm for Online Linear Optimization (OLO) with switching cost, which improves the existing suboptimal regret bound [ZCP22a] to the optimal rate.
  arXiv  Detail & Related papers  (2022-05-13T18:44:27Z)
• Budgeted Classification with Rejection: An Evolutionary Method with Multiple Objectives [0.0]
  Budgeted, sequential classifiers (BSCs) process inputs through a sequence of partial feature acquisition and evaluation steps. This allows for an efficient evaluation of inputs that prevents unneeded feature acquisition. We propose a problem-specific genetic algorithm to build budgeted, sequential classifiers with confidence-based reject options.
  arXiv  Detail & Related papers  (2022-05-01T22:05:16Z)
• SnAKe: Bayesian Optimization with Pathwise Exploration [9.807656882149319]
  We consider a novel setting where the expense of evaluating the function can increase significantly when making large input changes between iterations. This paper investigates the problem and introduces 'Sequential Bayesian Optimization via Adaptive Connecting Samples' (SnAKe) It provides a solution by considering future queries and preemptively building optimization paths that minimize input costs.
  arXiv  Detail & Related papers  (2022-01-31T19:42:56Z)
• Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs [28.254408148839644]
  We propose a non-myopic acquisition function that generalizes classical expected improvement to the setting of heterogeneous evaluation costs. Our acquisition function outperforms existing methods in a variety of synthetic and real problems.
  arXiv  Detail & Related papers  (2021-11-12T02:18:26Z)
• Batch Bayesian Optimization on Permutations using Acquisition Weighted Kernels [86.11176756341114]
  We introduce LAW, a new efficient batch acquisition method based on the determinantal point process. We provide a regret analysis for our method to gain insight in its theoretical properties. We evaluate the method on several standard problems involving permutations such as quadratic assignment.
  arXiv  Detail & Related papers  (2021-02-26T10:15:57Z)
• Particle Swarm Optimization: Fundamental Study and its Application to Optimization and to Jetty Scheduling Problems [0.0]
  The advantages of evolutionary algorithms with respect to traditional methods have been greatly discussed in the literature. While particle swarms share such advantages, they outperform evolutionary algorithms in that they require lower computational cost and easier implementation. This paper does not intend to study their tuning, general-purpose settings are taken from previous studies, and virtually the same algorithm is used to optimize a variety of notably different problems.
  arXiv  Detail & Related papers  (2021-01-25T02:06:30Z)
• Cost-Efficient Online Hyperparameter Optimization [94.60924644778558]
  We propose an online HPO algorithm that reaches human expert-level performance within a single run of the experiment. Our proposed online HPO algorithm reaches human expert-level performance within a single run of the experiment, while incurring only modest computational overhead compared to regular training.
  arXiv  Detail & Related papers  (2021-01-17T04:55:30Z)
• Constrained Multiagent Rollout and Multidimensional Assignment with the Auction Algorithm [0.0]
  We consider an extension of the rollout algorithm that applies to constrained deterministic dynamic programming. We show that if the base produces a feasible solution, the rollout algorithm has a cost improvement property. We show that the cost improvement property is maintained with an alternative implementation that has greatly reduced computational requirements.
  arXiv  Detail & Related papers  (2020-02-18T07:09:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.