Exploiting Separability in Multi-Scale Grey-Box Bayesian Optimization

• URL: http://arxiv.org/abs/2608.03045v1
• Date: Tue, 04 Aug 2026 02:53:54 GMT
• Title: Exploiting Separability in Multi-Scale Grey-Box Bayesian Optimization
• Abstract summary: We consider optimisation problems where the decision variables naturally partition into black-box variables and white-box variables.<n>We exploit this separability through a bilevel reformulation: an outer Bayesian optimization (BO) to optimize the objective as a function of black-box variables alone, while an inner problem solves the white-box subproblem via global optimization.<n>On a suite of 13 benchmark problems, bilevel BO achieves lower regret, with fewer iterations and wall clock time.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider grey-box optimization problems where the decision variables naturally partition into black-box variables (as arguments to an expensive black-box function) and white-box variables, governed by a set of explicit, closed-form equations that also depend on the output of the black-box function. We exploit this separability through a bilevel reformulation: an outer Bayesian optimization (BO) to optimize the scalar objective as a function of black-box variables alone, while an inner problem solves the white-box subproblem via global optimization. The Gaussian process surrogate used in BO is therefore defined rather than and white-box constraints are satisfied exactly whenever the inner optimizer converges to a feasible point---without penalty functions, chance constraints, or moment approximations. On a suite of 13 benchmark problems, bilevel BO achieves lower regret, with fewer iterations and wall clock time. This advantage is robust to initialization set size, exploration parameters, and inner-solver choice.

Related papers

• High Dimensional Bayesian Optimization using Lasso Variable Selection [9.051539805042651]
  We introduce a novel method that identifies important variables by estimating the length scales of Gaussian process kernels.<n>We demonstrate that our proposed method achieves cumulative regret with a sublinear growth rate in the worst case.<n> Experiments on high-dimensional synthetic functions and real-world problems show that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-04-02T13:54:04Z)
• Sharpness-Aware Black-Box Optimization [47.95184866255126]
  We propose a Sharpness-Aware Black-box Optimization (SABO) algorithm, which applies a sharpness-aware minimization strategy to improve the model generalization. Empirically, extensive experiments on the black-box prompt fine-tuning tasks demonstrate the effectiveness of the proposed SABO method in improving model generalization performance.
  arXiv  Detail & Related papers  (2024-10-16T11:08:06Z)
• Computing Star Discrepancies with Numerical Black-Box Optimization Algorithms [56.08144272945755]
  We compare 8 popular numerical black-box optimization algorithms on the $L_infty$ star discrepancy problem. We show that all used solvers perform very badly on a large majority of the instances. We suspect that state-of-the-art numerical black-box optimization techniques fail to capture the global structure of the problem.
  arXiv  Detail & Related papers  (2023-06-29T14:57:56Z)
• Bayesian Optimization of Expensive Nested Grey-Box Functions [11.523746174066702]
  We consider the problem of optimizing a grey-box objective function composed of both black-box and white-box functions. A general formulation for such grey-box problems is given, which covers the existing grey-box optimization formulations as special cases. We then design an optimism-driven algorithm to solve it.
  arXiv  Detail & Related papers  (2023-06-08T12:18:18Z)
• Bayesian Optimistic Optimisation with Exponentially Decaying Regret [58.02542541410322]
  The current practical BO algorithms have regret bounds ranging from $mathcalO(fraclogNsqrtN)$ to $mathcal O(e-sqrtN)$, where $N$ is the number of evaluations. This paper explores the possibility of improving the regret bound in the noiseless setting by intertwining concepts from BO and tree-based optimistic optimisation. We propose the BOO algorithm, a first practical approach which can achieve an exponential regret bound with order $mathcal O(N-sqrt
  arXiv  Detail & Related papers  (2021-05-10T13:07:44Z)
• Implicit differentiation for fast hyperparameter selection in non-smooth convex learning [87.60600646105696]
  We study first-order methods when the inner optimization problem is convex but non-smooth. We show that the forward-mode differentiation of proximal gradient descent and proximal coordinate descent yield sequences of Jacobians converging toward the exact Jacobian.
  arXiv  Detail & Related papers  (2021-05-04T17:31:28Z)
• Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information [78.78486761923855]
  In many real world problems, we want to infer some property of an expensive black-box function f, given a budget of T function evaluations. We present a procedure, InfoBAX, that sequentially chooses queries that maximize mutual information with respect to the algorithm's output. On these problems, InfoBAX uses up to 500 times fewer queries to f than required by the original algorithm.
  arXiv  Detail & Related papers  (2021-04-19T17:22:11Z)
• Adaptive Local Bayesian Optimization Over Multiple Discrete Variables [9.860437640748113]
  This paper describes the approach of team KAIST OSI in a step-wise manner, which outperforms the baseline algorithms by up to +20.39%. In a similar vein, we combine the methodology of Bayesian and multi-armed bandit,(MAB) approach to select the values with the consideration of the variable types. Empirical evaluations demonstrate that our method outperforms the existing methods across different tasks.
  arXiv  Detail & Related papers  (2020-12-07T07:51:23Z)
• Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes [43.19936635161588]
  We present a method for Bayesian optimization in a space which can operate well in a large space. Our algorithm shows satisfactory performance compared to existing methods.
  arXiv  Detail & Related papers  (2020-11-26T02:22:41Z)
• A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic [142.1492359556374]
  Bilevel optimization is a class of problems which exhibit a two-level structure. We propose a two-timescale approximation (TTSA) algorithm for tackling such a bilevel problem. We show that a two-timescale natural actor-critic policy optimization algorithm can be viewed as a special case of our TTSA framework.
  arXiv  Detail & Related papers  (2020-07-10T05:20:02Z)
• Incorporating Expert Prior in Bayesian Optimisation via Space Warping [54.412024556499254]
  In big search spaces the algorithm goes through several low function value regions before reaching the optimum of the function. One approach to subside this cold start phase is to use prior knowledge that can accelerate the optimisation. In this paper, we represent the prior knowledge about the function optimum through a prior distribution. The prior distribution is then used to warp the search space in such a way that space gets expanded around the high probability region of function optimum and shrinks around low probability region of optimum.
  arXiv  Detail & Related papers  (2020-03-27T06:18:49Z)
• DEFT-FUNNEL: an open-source global optimization solver for constrained grey-box and black-box problems [0.0]
  DEFT-FUNNEL is an open-source global optimization algorithm for general constrained grey-box and black-box problems. The code as well as the test sets used for experiments are available at the Github repository.
  arXiv  Detail & Related papers  (2019-12-29T11:43:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.