Revisiting Bayesian Optimization in the light of the COCO benchmark
- URL: http://arxiv.org/abs/2103.16649v1
- Date: Tue, 30 Mar 2021 19:45:18 GMT
- Title: Revisiting Bayesian Optimization in the light of the COCO benchmark
- Authors: Rodolphe Le Riche, Victor Picheny
- Abstract summary: This article reports a large investigation about the effects on the performance of (Gaussian process based) BO of common and less common design choices.
The code developed for this study makes the new version (v2.1.1) of the R package DiceOptim available on CRAN.
- Score: 1.4467794332678539
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: It is commonly believed that Bayesian optimization (BO) algorithms are highly
efficient for optimizing numerically costly functions. However, BO is not often
compared to widely different alternatives, and is mostly tested on narrow sets
of problems (multimodal, low-dimensional functions), which makes it difficult
to assess where (or if) they actually achieve state-of-the-art performance.
Moreover, several aspects in the design of these algorithms vary across
implementations without a clear recommendation emerging from current practices,
and many of these design choices are not substantiated by authoritative test
campaigns. This article reports a large investigation about the effects on the
performance of (Gaussian process based) BO of common and less common design
choices. The experiments are carried out with the established COCO (COmparing
Continuous Optimizers) software. It is found that a small initial budget, a
quadratic trend, high-quality optimization of the acquisition criterion bring
consistent progress. Using the GP mean as an occasional acquisition contributes
to a negligible additional improvement. Warping degrades performance. The
Mat\'ern 5/2 kernel is a good default but it may be surpassed by the
exponential kernel on irregular functions. Overall, the best EGO variants are
competitive or improve over state-of-the-art algorithms in dimensions less or
equal to 5 for multimodal functions. The code developed for this study makes
the new version (v2.1.1) of the R package DiceOptim available on CRAN. The
structure of the experiments by function groups allows to define priorities for
future research on Bayesian optimization.
Related papers
- A General Framework for User-Guided Bayesian Optimization [51.96352579696041]
We propose ColaBO, the first Bayesian-principled framework for prior beliefs beyond the typical kernel structure.
We empirically demonstrate ColaBO's ability to substantially accelerate optimization when the prior information is accurate, and to retain approximately default performance when it is misleading.
arXiv Detail & Related papers (2023-11-24T18:27:26Z) - Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB [0.40498500266986387]
We compare five state-of-the-art high-dimensional BO algorithms, with vanilla and CMA-ES, at increasing dimensionality, ranging from 10 to 60 variables.
Our results confirm the superiority of BO over CMA-ES for limited evaluation budgets.
arXiv Detail & Related papers (2023-03-02T01:14:15Z) - Generalizing Bayesian Optimization with Decision-theoretic Entropies [102.82152945324381]
We consider a generalization of Shannon entropy from work in statistical decision theory.
We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures.
We then show how alternative choices for the loss yield a flexible family of acquisition functions.
arXiv Detail & Related papers (2022-10-04T04:43:58Z) - Efficient computation of the Knowledge Gradient for Bayesian
Optimization [1.0497128347190048]
One-shot Hybrid KG is a new approach that combines several of the previously proposed ideas and is cheap to compute as well as powerful and efficient.
All experiments are implemented in BOTorch and show empirically drastically reduced computational overhead with equal or improved performance.
arXiv Detail & Related papers (2022-09-30T10:39:38Z) - MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation [0.01652719262940403]
optimisation problems often have multiple conflicting objectives that can be computationally and/or financially expensive.
Mono-surrogate Bayesian optimisation (BO) is a popular model-based approach for optimising such black-box functions.
We extend previous work on BO by density-ratio estimation (BORE) to the multi-objective setting.
arXiv Detail & Related papers (2022-03-31T09:27:59Z) - Bayesian Optimization over Permutation Spaces [30.650753803587794]
We propose and evaluate two algorithms for BO over Permutation Spaces (BOPS)
We theoretically analyze the performance of BOPS-T to show that their regret grows sub-linearly.
Our experiments on multiple synthetic and real-world benchmarks show that both BOPS-T and BOPS-H perform better than the state-of-the-art BO algorithm for spaces.
arXiv Detail & Related papers (2021-12-02T08:20:50Z) - Bayesian Optimistic Optimisation with Exponentially Decaying Regret [58.02542541410322]
The current practical BO algorithms have regret bounds ranging from $mathcalO(fraclogNsqrtN)$ to $mathcal O(e-sqrtN)$, where $N$ is the number of evaluations.
This paper explores the possibility of improving the regret bound in the noiseless setting by intertwining concepts from BO and tree-based optimistic optimisation.
We propose the BOO algorithm, a first practical approach which can achieve an exponential regret bound with order $mathcal O(N-sqrt
arXiv Detail & Related papers (2021-05-10T13:07:44Z) - Bayesian Algorithm Execution: Estimating Computable Properties of
Black-box Functions Using Mutual Information [78.78486761923855]
In many real world problems, we want to infer some property of an expensive black-box function f, given a budget of T function evaluations.
We present a procedure, InfoBAX, that sequentially chooses queries that maximize mutual information with respect to the algorithm's output.
On these problems, InfoBAX uses up to 500 times fewer queries to f than required by the original algorithm.
arXiv Detail & Related papers (2021-04-19T17:22:11Z) - Recent Theoretical Advances in Non-Convex Optimization [56.88981258425256]
Motivated by recent increased interest in analysis of optimization algorithms for non- optimization in deep networks and other problems in data, we give an overview of recent results of theoretical optimization algorithms for non- optimization.
arXiv Detail & Related papers (2020-12-11T08:28:51Z) - Bayesian Optimization with a Prior for the Optimum [41.41323474440455]
We introduce Bayesian Optimization with a Prior for the Optimum (BOPrO)
BOPrO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance.
We show that BOPrO is around 6.67x faster than state-of-the-art methods on a common suite of benchmarks.
arXiv Detail & Related papers (2020-06-25T17:49:24Z) - Incorporating Expert Prior in Bayesian Optimisation via Space Warping [54.412024556499254]
In big search spaces the algorithm goes through several low function value regions before reaching the optimum of the function.
One approach to subside this cold start phase is to use prior knowledge that can accelerate the optimisation.
In this paper, we represent the prior knowledge about the function optimum through a prior distribution.
The prior distribution is then used to warp the search space in such a way that space gets expanded around the high probability region of function optimum and shrinks around low probability region of optimum.
arXiv Detail & Related papers (2020-03-27T06:18:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.