Bayesian Quantile and Expectile Optimisation
- URL: http://arxiv.org/abs/2001.04833v2
- Date: Thu, 7 Jul 2022 21:42:39 GMT
- Title: Bayesian Quantile and Expectile Optimisation
- Authors: Victor Picheny, Henry Moss, L\'eonard Torossian and Nicolas Durrande
- Abstract summary: We propose new variational models for Bayesian quantile and expectile regression that are well-suited for heteroscedastic noise settings.
Our strategies can directly optimise for the quantile and expectile, without requiring replicating observations or assuming a parametric form for the noise.
As illustrated in the experimental section, the proposed approach clearly outperforms the state of the art in the heteroscedastic, non-Gaussian case.
- Score: 3.3878745408530833
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Bayesian optimisation (BO) is widely used to optimise stochastic black box
functions. While most BO approaches focus on optimising conditional
expectations, many applications require risk-averse strategies and alternative
criteria accounting for the distribution tails need to be considered. In this
paper, we propose new variational models for Bayesian quantile and expectile
regression that are well-suited for heteroscedastic noise settings. Our models
consist of two latent Gaussian processes accounting respectively for the
conditional quantile (or expectile) and the scale parameter of an asymmetric
likelihood functions. Furthermore, we propose two BO strategies based on
max-value entropy search and Thompson sampling, that are tailored to such
models and that can accommodate large batches of points. Contrary to existing
BO approaches for risk-averse optimisation, our strategies can directly
optimise for the quantile and expectile, without requiring replicating
observations or assuming a parametric form for the noise. As illustrated in the
experimental section, the proposed approach clearly outperforms the state of
the art in the heteroscedastic, non-Gaussian case.
Related papers
- Poisson Process for Bayesian Optimization [126.51200593377739]
We propose a ranking-based surrogate model based on the Poisson process and introduce an efficient BO framework, namely Poisson Process Bayesian Optimization (PoPBO)
Compared to the classic GP-BO method, our PoPBO has lower costs and better robustness to noise, which is verified by abundant experiments.
arXiv Detail & Related papers (2024-02-05T02:54:50Z) - Simulation Based Bayesian Optimization [0.6526824510982799]
This paper introduces Simulation Based Bayesian Optimization (SBBO) as a novel approach to optimizing acquisition functions.
SBBO allows the use of surrogate models tailored for spaces with discrete variables.
We demonstrate empirically the effectiveness of SBBO method using various choices of surrogate models.
arXiv Detail & Related papers (2024-01-19T16:56:11Z) - Pseudo-Bayesian Optimization [7.556071491014536]
We study an axiomatic framework that elicits the minimal requirements to guarantee black-box optimization convergence.
We show how using simple local regression, and a suitable "randomized prior" construction to quantify uncertainty, not only guarantees convergence but also consistently outperforms state-of-the-art benchmarks.
arXiv Detail & Related papers (2023-10-15T07:55:28Z) - Learning Regions of Interest for Bayesian Optimization with Adaptive
Level-Set Estimation [84.0621253654014]
We propose a framework, called BALLET, which adaptively filters for a high-confidence region of interest.
We show theoretically that BALLET can efficiently shrink the search space, and can exhibit a tighter regret bound than standard BO.
arXiv Detail & Related papers (2023-07-25T09:45:47Z) - Bayesian Optimization with Conformal Prediction Sets [44.565812181545645]
Conformal prediction is an uncertainty quantification method with coverage guarantees even for misspecified models.
We propose conformal Bayesian optimization, which directs queries towards regions of search space where the model predictions have guaranteed validity.
In many cases we find that query coverage can be significantly improved without harming sample-efficiency.
arXiv Detail & Related papers (2022-10-22T17:01:05Z) - Surrogate modeling for Bayesian optimization beyond a single Gaussian
process [62.294228304646516]
We propose a novel Bayesian surrogate model to balance exploration with exploitation of the search space.
To endow function sampling with scalability, random feature-based kernel approximation is leveraged per GP model.
To further establish convergence of the proposed EGP-TS to the global optimum, analysis is conducted based on the notion of Bayesian regret.
arXiv Detail & Related papers (2022-05-27T16:43:10Z) - Risk-averse Heteroscedastic Bayesian Optimization [45.12421486836736]
We propose a novel risk-averse heteroscedastic Bayesian optimization algorithm (RAHBO)
RAHBO aims to identify a solution with high return and low noise variance, while learning the noise distribution on the fly.
We provide a robust rule to report the final decision point for applications where only a single solution must be identified.
arXiv Detail & Related papers (2021-11-05T17:38:34Z) - Variational Refinement for Importance Sampling Using the Forward
Kullback-Leibler Divergence [77.06203118175335]
Variational Inference (VI) is a popular alternative to exact sampling in Bayesian inference.
Importance sampling (IS) is often used to fine-tune and de-bias the estimates of approximate Bayesian inference procedures.
We propose a novel combination of optimization and sampling techniques for approximate Bayesian inference.
arXiv Detail & Related papers (2021-06-30T11:00:24Z) - High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise [51.31435087414348]
It is essential to theoretically guarantee that algorithms provide small objective residual with high probability.
Existing methods for non-smooth convex optimization have complexity bounds with dependence on confidence level.
We propose novel stepsize rules for two methods with gradient clipping.
arXiv Detail & Related papers (2021-06-10T17:54:21Z) - Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution.
Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
arXiv Detail & Related papers (2021-06-07T17:44:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.