Abstract: This paper describes a general-purpose extension of max-value entropy search,
a popular approach for Bayesian Optimisation (BO). A novel approximation is
proposed for the information gain -- an information-theoretic quantity central
to solving a range of BO problems, including noisy, multi-fidelity and batch
optimisations across both continuous and highly-structured discrete spaces.
Previously, these problems have been tackled separately within
information-theoretic BO, each requiring a different sophisticated
approximation scheme, except for batch BO, for which no
computationally-lightweight information-theoretic approach has previously been
proposed. GIBBON (General-purpose Information-Based Bayesian OptimisatioN)
provides a single principled framework suitable for all the above,
out-performing existing approaches whilst incurring substantially lower
computational overheads. In addition, GIBBON does not require the problem's
search space to be Euclidean and so is the first high-performance yet
computationally light-weight acquisition function that supports batch BO over
general highly structured input spaces like molecular search and gene design.
Moreover, our principled derivation of GIBBON yields a natural interpretation
of a popular batch BO heuristic based on determinantal point processes.
Finally, we analyse GIBBON across a suite of synthetic benchmark tasks, a
molecular search loop, and as part of a challenging batch multi-fidelity
framework for problems with controllable experimental noise.