Mean-Variance Analysis in Bayesian Optimization under Uncertainty

• URL: http://arxiv.org/abs/2009.08166v1
• Date: Thu, 17 Sep 2020 09:21:46 GMT
• Title: Mean-Variance Analysis in Bayesian Optimization under Uncertainty
• Authors: Shogo Iwazaki, Yu Inatsu, Ichiro Takeuchi
• Abstract summary: We consider active learning (AL) in an uncertain environment in which trade-off between multiple risk measures need to be considered. We show the effectiveness of the proposed AL algorithms through theoretical analysis and numerical experiments.
• Score: 23.39754660544729
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider active learning (AL) in an uncertain environment in which trade-off between multiple risk measures need to be considered. As an AL problem in such an uncertain environment, we study Mean-Variance Analysis in Bayesian Optimization (MVA-BO) setting. Mean-variance analysis was developed in the field of financial engineering and has been used to make decisions that take into account the trade-off between the average and variance of investment uncertainty. In this paper, we specifically focus on BO setting with an uncertain component and consider multi-task, multi-objective, and constrained optimization scenarios for the mean-variance trade-off of the uncertain component. When the target blackbox function is modeled by Gaussian Process (GP), we derive the bounds of the two risk measures and propose AL algorithm for each of the above three problems based on the risk measure bounds. We show the effectiveness of the proposed AL algorithms through theoretical analysis and numerical experiments.

Related papers

• Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization [59.758009422067]
  We consider the problem of quantifying uncertainty over expected cumulative rewards in model-based reinforcement learning. We propose a new uncertainty Bellman equation (UBE) whose solution converges to the true posterior variance over values. We introduce a general-purpose policy optimization algorithm, Q-Uncertainty Soft Actor-Critic (QU-SAC) that can be applied for either risk-seeking or risk-averse policy optimization.
  arXiv  Detail & Related papers  (2023-12-07T15:55:58Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Nonlinear Distributionally Robust Optimization [5.0490573482829335]
  This article focuses on a class of distributionally robust optimization (DRO) problems where the objective function is potentially nonlinear in the distribution. We propose an alternative notion for the derivative and corresponding smoothness based on Gateaux (G)-derivative for generic risk measures. We use the set-up of the FW algorithm to devise a methodology to compute a saddle point of the nonlinear DRO problem.
  arXiv  Detail & Related papers  (2023-06-05T19:22:02Z)
• Bounding Box-based Multi-objective Bayesian Optimization of Risk Measures under Input Uncertainty [21.056363101777052]
  We propose a novel multi-objective Bayesian optimization (MOBO) method to efficiently identify the Pareto front (PF) defined by risk measures for black-box functions under the presence of input uncertainty (IU) The basic idea of the proposed method is to assume a Gaussian process (GP) model for the black-box function and to construct high-probability bounding boxes for the risk measures using the GP model. We prove that the algorithm can return an arbitrary-accurate solution in a finite number of iterations with high probability, for various risk measures such as Bayes risk, worst-case risk, and value-
  arXiv  Detail & Related papers  (2023-01-27T08:25:45Z)
• Uncertainty-Driven Action Quality Assessment [67.20617610820857]
  We propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to capture the diversity among multiple judge scores. We generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss. Our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.
  arXiv  Detail & Related papers  (2022-07-29T07:21:15Z)
• Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learning [12.022303947412917]
  This paper aims at optimizing the mean-semivariance criterion in reinforcement learning w.r.t. steady rewards. We reveal that the MSV problem can be solved by iteratively solving a sequence of RL problems with a policy-dependent reward function. We propose two on-policy algorithms based on the policy gradient theory and the trust region method.
  arXiv  Detail & Related papers  (2022-06-15T08:32:53Z)
• Optimal variance-reduced stochastic approximation in Banach spaces [114.8734960258221]
  We study the problem of estimating the fixed point of a contractive operator defined on a separable Banach space. We establish non-asymptotic bounds for both the operator defect and the estimation error.
  arXiv  Detail & Related papers  (2022-01-21T02:46:57Z)
• Risk-Averse Bayes-Adaptive Reinforcement Learning [3.5289688061934963]
  We pose the problem of optimising the conditional value at risk (CVaR) of the total return in Bayes-adaptive Markov decision processes (MDPs) We show that a policy optimising CVaR in this setting is risk-averse to both the parametric uncertainty due to the prior distribution over MDPs, and the internal uncertainty due to the inherentity of MDPs. Our experiments demonstrate that our approach significantly outperforms baseline approaches for this problem.
  arXiv  Detail & Related papers  (2021-02-10T22:34:33Z)
• Large-Scale Methods for Distributionally Robust Optimization [53.98643772533416]
  We prove that our algorithms require a number of evaluations gradient independent of training set size and number of parameters. Experiments on MNIST and ImageNet confirm the theoretical scaling of our algorithms, which are 9--36 times more efficient than full-batch methods.
  arXiv  Detail & Related papers  (2020-10-12T17:41:44Z)
• Thompson Sampling Algorithms for Mean-Variance Bandits [97.43678751629189]
  We develop Thompson Sampling-style algorithms for mean-variance MAB. We also provide comprehensive regret analyses for Gaussian and Bernoulli bandits. Our algorithms significantly outperform existing LCB-based algorithms for all risk tolerances.
  arXiv  Detail & Related papers  (2020-02-01T15:33:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.