A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences

• URL: http://arxiv.org/abs/2406.04739v2
• Date: Mon, 04 Nov 2024 10:02:09 GMT
• Title: A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences
• Authors: Miguel González-Duque, Richard Michael, Simon Bartels, Yevgen Zainchkovskyy, Søren Hauberg, Wouter Boomsma,
• Abstract summary: optimizing discrete black-box functions is key in several domains, e.g. protein engineering and drug design. We develop a unified framework to test a vast array of high-dimensional Bayesian optimization methods and a collection of standardized black-box functions. These two components of the benchmark are each supported by flexible, scalable, and easily extendable software libraries.
• Score: 12.248793682283964
• License:
• Abstract: Optimizing discrete black-box functions is key in several domains, e.g. protein engineering and drug design. Due to the lack of gradient information and the need for sample efficiency, Bayesian optimization is an ideal candidate for these tasks. Several methods for high-dimensional continuous and categorical Bayesian optimization have been proposed recently. However, our survey of the field reveals highly heterogeneous experimental set-ups across methods and technical barriers for the replicability and application of published algorithms to real-world tasks. To address these issues, we develop a unified framework to test a vast array of high-dimensional Bayesian optimization methods and a collection of standardized black-box functions representing real-world application domains in chemistry and biology. These two components of the benchmark are each supported by flexible, scalable, and easily extendable software libraries (poli and poli-baselines), allowing practitioners to readily incorporate new optimization objectives or discrete optimizers. Project website: https://machinelearninglifescience.github.io/hdbo_benchmark

Related papers

• Localized Zeroth-Order Prompt Optimization [54.964765668688806]
  We propose a novel algorithm, namely localized zeroth-order prompt optimization (ZOPO) ZOPO incorporates a Neural Tangent Kernel-based derived Gaussian process into standard zeroth-order optimization for an efficient search of well-performing local optima in prompt optimization. Remarkably, ZOPO outperforms existing baselines in terms of both the optimization performance and the query efficiency.
  arXiv  Detail & Related papers  (2024-03-05T14:18:15Z)
• Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties [49.351577714596544]
  We propose a human-AI collaborative Bayesian framework to incorporate expert preferences about unmeasured abstract properties into surrogate modeling. We provide an efficient strategy that can also handle any incorrect/misleading expert bias in preferential judgments.
  arXiv  Detail & Related papers  (2024-02-27T09:23:13Z)
• High-dimensional Bayesian Optimization with Group Testing [7.12295305987761]
  We propose a group testing approach to identify active variables to facilitate efficient optimization in high-dimensional domains. The proposed algorithm, Group Testing Bayesian Optimization (GTBO), first runs a testing phase where groups of variables are systematically selected and tested. In the second phase, GTBO guides optimization by placing more importance on the active dimensions.
  arXiv  Detail & Related papers  (2023-10-05T12:52:27Z)
• Transfer Learning for Bayesian Optimization: A Survey [29.229660973338145]
  Black-box optimization is a powerful tool that models and optimize such expensive "black-box" functions. Researchers in the BO community propose to incorporate the spirit of transfer learning to accelerate optimization process.
  arXiv  Detail & Related papers  (2023-02-12T14:37:25Z)
• An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization [78.36413169647408]
  We study the effectiveness of various ZO optimization methods for optimizing molecular objectives. We show the advantages of ZO sign-based gradient descent (ZO-signGD) We demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
  arXiv  Detail & Related papers  (2022-10-27T01:58:10Z)
• Generalizing Bayesian Optimization with Decision-theoretic Entropies [102.82152945324381]
  We consider a generalization of Shannon entropy from work in statistical decision theory. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures. We then show how alternative choices for the loss yield a flexible family of acquisition functions.
  arXiv  Detail & Related papers  (2022-10-04T04:43:58Z)
• Efficient Non-Parametric Optimizer Search for Diverse Tasks [93.64739408827604]
  We present the first efficient scalable and general framework that can directly search on the tasks of interest. Inspired by the innate tree structure of the underlying math expressions, we re-arrange the spaces into a super-tree. We adopt an adaptation of the Monte Carlo method to tree search, equipped with rejection sampling and equivalent- form detection.
  arXiv  Detail & Related papers  (2022-09-27T17:51:31Z)
• A Robust Multi-Objective Bayesian Optimization Framework Considering Input Uncertainty [0.0]
  In real-life applications like engineering design, the designer often wants to take multiple objectives as well as input uncertainty into account. We introduce a novel Bayesian optimization framework to efficiently perform multi-objective optimization considering input uncertainty.
  arXiv  Detail & Related papers  (2022-02-25T17:45:26Z)
• Batched Data-Driven Evolutionary Multi-Objective Optimization Based on Manifold Interpolation [6.560512252982714]
  We propose a framework for implementing batched data-driven evolutionary multi-objective optimization. It is so general that any off-the-shelf evolutionary multi-objective optimization algorithms can be applied in a plug-in manner. Our proposed framework is featured with a faster convergence and a stronger resilience to various PF shapes.
  arXiv  Detail & Related papers  (2021-09-12T23:54:26Z)
• Bayesian Variational Optimization for Combinatorial Spaces [0.0]
  Broad applications include the study of molecules, proteins, DNA, device structures and quantum circuit designs. A on optimization over categorical spaces is needed to find optimal or pareto-optimal solutions. We introduce a variational Bayesian optimization method that combines variational optimization and continuous relaxations.
  arXiv  Detail & Related papers  (2020-11-03T20:56:13Z)
• Incorporating Expert Prior in Bayesian Optimisation via Space Warping [54.412024556499254]
  In big search spaces the algorithm goes through several low function value regions before reaching the optimum of the function. One approach to subside this cold start phase is to use prior knowledge that can accelerate the optimisation. In this paper, we represent the prior knowledge about the function optimum through a prior distribution. The prior distribution is then used to warp the search space in such a way that space gets expanded around the high probability region of function optimum and shrinks around low probability region of optimum.
  arXiv  Detail & Related papers  (2020-03-27T06:18:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.