On the Implementation of a Bayesian Optimization Framework for Interconnected Systems

• URL: http://arxiv.org/abs/2501.00967v1
• Date: Wed, 01 Jan 2025 21:58:19 GMT
• Title: On the Implementation of a Bayesian Optimization Framework for Interconnected Systems
• Authors: Leonardo D. González, Victor M. Zavala,
• Abstract summary: We provide a detailed implementation of a recently proposed grey-box BO paradigm, BOIS, that uses adaptive linearizations of $f$ to obtain analytical expressions. We show that the BOIS approach enables the exploitation of structural knowledge, such as that arising in interconnected systems. Our results indicate BOIS performs as well as or better than existing grey-box methods, while also being less computationally intensive.
• Score: 0.0
• License:
• Abstract: Bayesian optimization (BO) is an effective paradigm for the optimization of expensive-to-sample systems. Standard BO learns the performance of a system $f(x)$ by using a Gaussian Process (GP) model; this treats the system as a black-box and limits its ability to exploit available structural knowledge (e.g., physics and sparse interconnections in a complex system). Grey-box modeling, wherein the performance function is treated as a composition of known and unknown intermediate functions $f(x, y(x))$ (where $y(x)$ is a GP model) offers a solution to this limitation; however, generating an analytical probability density for $f$ from the Gaussian density of $y(x)$ is often an intractable problem (e.g., when $f$ is nonlinear). Previous work has handled this issue by using sampling techniques or by solving an auxiliary problem over an augmented space where the values of $y(x)$ are constrained by confidence intervals derived from the GP models; such solutions are computationally intensive. In this work, we provide a detailed implementation of a recently proposed grey-box BO paradigm, BOIS, that uses adaptive linearizations of $f$ to obtain analytical expressions for the statistical moments of the composite function. We show that the BOIS approach enables the exploitation of structural knowledge, such as that arising in interconnected systems as well as systems that embed multiple GP models and combinations of physics and GP models. We benchmark the effectiveness of BOIS against standard BO and existing grey-box BO algorithms using a pair of case studies focused on chemical process optimization and design. Our results indicate that BOIS performs as well as or better than existing grey-box methods, while also being less computationally intensive.

Related papers

• Near-Optimal Online Learning for Multi-Agent Submodular Coordination: Tight Approximation and Communication Efficiency [52.60557300927007]
  We present a $textbfMA-OSMA$ algorithm to transfer the discrete submodular problem into a continuous optimization. We also introduce a projection-free $textbfMA-OSEA$ algorithm, which effectively utilizes the KL divergence by mixing a uniform distribution. Our algorithms significantly improve the $(frac11+c)$-approximation provided by the state-of-the-art OSG algorithm.
  arXiv  Detail & Related papers  (2025-02-07T15:57:56Z)
• Poisson Process for Bayesian Optimization [126.51200593377739]
  We propose a ranking-based surrogate model based on the Poisson process and introduce an efficient BO framework, namely Poisson Process Bayesian Optimization (PoPBO) Compared to the classic GP-BO method, our PoPBO has lower costs and better robustness to noise, which is verified by abundant experiments.
  arXiv  Detail & Related papers  (2024-02-05T02:54:50Z)
• BOIS: Bayesian Optimization of Interconnected Systems [0.0]
  We introduce a new paradigm which allows for the efficient use of composite functions in BO. We show that this simple approach (which we call BOIS) enables the exploitation of structural knowledge. Our results indicate that BOIS achieves performance gains and accurately captures the statistics of composite functions.
  arXiv  Detail & Related papers  (2023-11-19T06:44:13Z)
• Joint Composite Latent Space Bayesian Optimization [15.262166538890243]
  We introduce Joint Composite Latent Space Bayesian Optimization (JoCo) JoCo is a novel framework that jointly trains neural network encoders and probabilistic models to adaptively compress high-dimensional input and output spaces into manageable latent representations. This enables viable BO on these compressed representations, allowing JoCo to outperform other state-of-the-art methods in high-dimensional BO on a wide variety of simulated and real-world problems.
  arXiv  Detail & Related papers  (2023-11-03T19:53:37Z)
• Polynomial-Model-Based Optimization for Blackbox Objectives [0.0]
  Black-box optimization seeks to find optimal parameters for systems such that a pre-defined objective function is minimized. PMBO is a novel blackbox that finds the minimum by fitting a surrogate to the objective function. PMBO is benchmarked against other state-of-the-art algorithms for a given set of artificial, analytical functions.
  arXiv  Detail & Related papers  (2023-09-01T14:11:03Z)
• Bayesian Optimization for Function Compositions with Applications to Dynamic Pricing [0.0]
  We propose a practical BO method of function compositions where the form of the composition is known but the constituent functions are expensive to evaluate. We demonstrate a novel application to dynamic pricing in revenue management when the underlying demand function is expensive to evaluate.
  arXiv  Detail & Related papers  (2023-03-21T15:45:06Z)
• Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization [88.0031283949404]
  Many real-world problems have complicated non functional constraints and use a large number of data points. Our proposed method outperforms an existing method with the previously best-known result.
  arXiv  Detail & Related papers  (2022-12-19T14:48:54Z)
• Model-based Causal Bayesian Optimization [78.120734120667]
  We propose model-based causal Bayesian optimization (MCBO) MCBO learns a full system model instead of only modeling intervention-reward pairs. Unlike in standard Bayesian optimization, our acquisition function cannot be evaluated in closed form.
  arXiv  Detail & Related papers  (2022-11-18T14:28:21Z)
• Surrogate modeling for Bayesian optimization beyond a single Gaussian process [62.294228304646516]
  We propose a novel Bayesian surrogate model to balance exploration with exploitation of the search space. To endow function sampling with scalability, random feature-based kernel approximation is leveraged per GP model. To further establish convergence of the proposed EGP-TS to the global optimum, analysis is conducted based on the notion of Bayesian regret.
  arXiv  Detail & Related papers  (2022-05-27T16:43:10Z)
• MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation [0.01652719262940403]
  optimisation problems often have multiple conflicting objectives that can be computationally and/or financially expensive. Mono-surrogate Bayesian optimisation (BO) is a popular model-based approach for optimising such black-box functions. We extend previous work on BO by density-ratio estimation (BORE) to the multi-objective setting.
  arXiv  Detail & Related papers  (2022-03-31T09:27:59Z)
• Misspecified Gaussian Process Bandit Optimization [59.30399661155574]
  Kernelized bandit algorithms have shown strong empirical and theoretical performance for this problem. We introduce a emphmisspecified kernelized bandit setting where the unknown function can be $epsilon$--uniformly approximated by a function with a bounded norm in some Reproducing Kernel Hilbert Space (RKHS) We show that our algorithm achieves optimal dependence on $epsilon$ with no prior knowledge of misspecification.
  arXiv  Detail & Related papers  (2021-11-09T09:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.