Bayesian Optimization with Missing Inputs
- URL: http://arxiv.org/abs/2006.10948v1
- Date: Fri, 19 Jun 2020 03:56:27 GMT
- Title: Bayesian Optimization with Missing Inputs
- Authors: Phuc Luong, Dang Nguyen, Sunil Gupta, Santu Rana, and Svetha Venkatesh
- Abstract summary: We develop a new acquisition function based on the well-known Upper Confidence Bound (UCB) acquisition function.
We conduct comprehensive experiments on both synthetic and real-world applications to show the usefulness of our method.
- Score: 53.476096769837724
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Bayesian optimization (BO) is an efficient method for optimizing expensive
black-box functions. In real-world applications, BO often faces a major problem
of missing values in inputs. The missing inputs can happen in two cases. First,
the historical data for training BO often contain missing values. Second, when
performing the function evaluation (e.g. computing alloy strength in a heat
treatment process), errors may occur (e.g. a thermostat stops working) leading
to an erroneous situation where the function is computed at a random unknown
value instead of the suggested value. To deal with this problem, a common
approach just simply skips data points where missing values happen. Clearly,
this naive method cannot utilize data efficiently and often leads to poor
performance. In this paper, we propose a novel BO method to handle missing
inputs. We first find a probability distribution of each missing value so that
we can impute the missing value by drawing a sample from its distribution. We
then develop a new acquisition function based on the well-known Upper
Confidence Bound (UCB) acquisition function, which considers the uncertainty of
imputed values when suggesting the next point for function evaluation. We
conduct comprehensive experiments on both synthetic and real-world applications
to show the usefulness of our method.
Related papers
- Online non-parametric likelihood-ratio estimation by Pearson-divergence
functional minimization [55.98760097296213]
We introduce a new framework for online non-parametric LRE (OLRE) for the setting where pairs of iid observations $(x_t sim p, x'_t sim q)$ are observed over time.
We provide theoretical guarantees for the performance of the OLRE method along with empirical validation in synthetic experiments.
arXiv Detail & Related papers (2023-11-03T13:20:11Z) - Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels [57.46832672991433]
We propose a novel equation discovery method based on Kernel learning and BAyesian Spike-and-Slab priors (KBASS)
We use kernel regression to estimate the target function, which is flexible, expressive, and more robust to data sparsity and noises.
We develop an expectation-propagation expectation-maximization algorithm for efficient posterior inference and function estimation.
arXiv Detail & Related papers (2023-10-09T03:55:09Z) - Provably Efficient Bayesian Optimization with Unknown Gaussian Process Hyperparameter Estimation [44.53678257757108]
We propose a new BO method that can sub-linearly converge to the objective function's global optimum.
Our method uses a multi-armed bandit technique (EXP3) to add random data points to the BO process.
We demonstrate empirically that our method outperforms existing approaches on various synthetic and real-world problems.
arXiv Detail & Related papers (2023-06-12T03:35:45Z) - Bayesian Optimization under Stochastic Delayed Feedback [36.16843889404038]
Existing BO methods assume that the function evaluation (feedback) is available to the learner immediately or after a fixed delay.
We propose algorithms with sub-linear regret guarantees that efficiently address the dilemma of selecting new function queries while waiting for delayed feedback.
arXiv Detail & Related papers (2022-06-19T07:34:08Z) - A Huber loss-based super learner with applications to healthcare
expenditures [0.0]
We propose a super learner based on the Huber loss, a "robust" loss function that combines squared error loss with absolute loss to downweight.
We show that the proposed method can be used both directly to optimize Huber risk, as well as in finite-sample settings.
arXiv Detail & Related papers (2022-05-13T19:57:50Z) - Signal to Noise Ratio Loss Function [0.0]
This work proposes a new loss function targeting classification problems.
We derive a series of tightest upper and lower bounds for the probability of a random variable in a given interval.
A lower bound is proposed for the probability of a true positive for a parametric classification problem.
arXiv Detail & Related papers (2021-10-23T18:44:32Z) - Efficient Exploration in Binary and Preferential Bayesian Optimization [0.5076419064097732]
We show that it is important for BO algorithms to distinguish between different types of uncertainty.
We propose several new acquisition functions that outperform state-of-the-art BO functions.
arXiv Detail & Related papers (2021-10-18T14:44:34Z) - Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
Learning from implicit feedback is challenging because of the difficult nature of the one-class problem.
Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem.
We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
arXiv Detail & Related papers (2021-05-11T03:38:16Z) - Bayesian Algorithm Execution: Estimating Computable Properties of
Black-box Functions Using Mutual Information [78.78486761923855]
In many real world problems, we want to infer some property of an expensive black-box function f, given a budget of T function evaluations.
We present a procedure, InfoBAX, that sequentially chooses queries that maximize mutual information with respect to the algorithm's output.
On these problems, InfoBAX uses up to 500 times fewer queries to f than required by the original algorithm.
arXiv Detail & Related papers (2021-04-19T17:22:11Z) - Piecewise Linear Regression via a Difference of Convex Functions [50.89452535187813]
We present a new piecewise linear regression methodology that utilizes fitting a difference of convex functions (DC functions) to the data.
We empirically validate the method, showing it to be practically implementable, and to have comparable performance to existing regression/classification methods on real-world datasets.
arXiv Detail & Related papers (2020-07-05T18:58:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.