Bayesian Optimization of Partially Known Systems using Hybrid Models

• URL: http://arxiv.org/abs/2603.11199v1
• Date: Wed, 11 Mar 2026 18:13:26 GMT
• Title: Bayesian Optimization of Partially Known Systems using Hybrid Models
• Abstract summary: We present a hybrid model-based BO formulation that combines the iterative Bayesian learning of BO with partially known mechanistic physical models.<n>In an in-silico optimization of a single-stage distillation, the hybrid BO model based on mass conservation laws yields significantly better designs than a standard BO loop.
• Score: 37.458438388741214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian optimization (BO) has gained attention as an efficient algorithm for black-box optimization of expensive-to-evaluate systems, where the BO algorithm iteratively queries the system and suggests new trials based on a probabilistic model fitted to previous samples. Still, the standard BO loop may require a prohibitively large number of experiments to converge to the optimum, especially for high-dimensional and nonlinear systems. We present a hybrid model-based BO formulation that combines the iterative Bayesian learning of BO with partially known mechanistic physical models. Instead of learning a direct mapping from inputs to the objective, we write all known equations for a physics-based model and infer expressions for variables missing equations using a probabilistic model, in our case, a Gaussian process (GP). The final formulation then includes the GP as a constraint in the hybrid model, thereby allowing other physics-based nonlinear and implicit model constraints. This hybrid model formulation yields a constrained, nonlinear stochastic program, which we discretize using the sample-average approximation. In an in-silico optimization of a single-stage distillation, the hybrid BO model based on mass conservation laws yields significantly better designs than a standard BO loop. Furthermore, the hybrid model converges in as few as one iteration, depending on the initial samples, whereas, the standard BO does not converge within 25 for any of the seeds. Overall, the proposed hybrid BO scheme presents a promising optimization method for partially known systems, leveraging the strengths of both mechanistic modeling and data-driven optimization.

Related papers

• Generative Bayesian Optimization: Generative Models as Acquisition Functions [13.375195089272685]
  We present a strategy for turning generative models into candidate solution samplers for batch optimization (BO)<n>The use of generative models for BO enables large batch scaling as generative sampling, optimization of non-continuous design spaces, and high-dimensional and high-dimensional design.
  arXiv  Detail & Related papers  (2025-10-29T07:42:25Z)
• Composition and Alignment of Diffusion Models using Constrained Learning [79.36736636241564]
  Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions.<n>Two commonly used methods are: (i) alignment, which involves fine-tuning a diffusion model to align it with a reward; and (ii) composition, which combines several pre-trained diffusion models, each emphasizing a desirable attribute in the generated outputs.<n>We propose a constrained optimization framework that unifies alignment and composition of diffusion models by enforcing that the aligned model satisfies reward constraints and/or remains close to (potentially multiple) pre-trained models.
  arXiv  Detail & Related papers  (2025-08-26T15:06:30Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• An Iterative Bayesian Approach for System Identification based on Linear Gaussian Models [70.75865595489691]
  We tackle the problem of system identification, where we select inputs, observe the corresponding outputs from the true system, and optimize the parameters of our model to best fit the data.<n>Our approach only requires input-output data from the system and first-order information of the model with respect to the parameters.
  arXiv  Detail & Related papers  (2025-01-28T01:57:51Z)
• Go With the Flow: Fast Diffusion for Gaussian Mixture Models [13.836464287004505]
  Schrodinger Bridges (SBs) are diffusion processes that steer in finite time, a given initial distribution to another final one while minimizing a suitable cost functional.<n>We propose an analytic parametrization of a set of feasible policies for solving low-to-dimensional problems.<n>We showcase the potential of this approach in low-to-dimensional problems such as image-to- latent translation in the space of an autoencoder, learning of cellular dynamics using multi-marginal momentum SBs, and various other examples.
  arXiv  Detail & Related papers  (2024-12-12T08:40:22Z)
• Simulation Based Bayesian Optimization [0.0]
  This paper introduces Simulation Based Bayesian Optimization (SBBO) as a novel approach to optimizing acquisition functions.<n>GPs are commonly used as the surrogate model as they offer analytical access to posterior predictive distributions.<n>We demonstrate empirically the effectiveness of SBBO using various choices of surrogate models.
  arXiv  Detail & Related papers  (2024-01-19T16:56:11Z)
• Predictive Modeling through Hyper-Bayesian Optimization [60.586813904500595]
  We propose a novel way of integrating model selection and BO for the single goal of reaching the function optima faster. The algorithm moves back and forth between BO in the model space and BO in the function space, where the goodness of the recommended model is captured. In addition to improved sample efficiency, the framework outputs information about the black-box function.
  arXiv  Detail & Related papers  (2023-08-01T04:46:58Z)
• Model-based Causal Bayesian Optimization [78.120734120667]
  We propose model-based causal Bayesian optimization (MCBO) MCBO learns a full system model instead of only modeling intervention-reward pairs. Unlike in standard Bayesian optimization, our acquisition function cannot be evaluated in closed form.
  arXiv  Detail & Related papers  (2022-11-18T14:28:21Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• Recent advances in Bayesian optimization with applications to parameter reconstruction in optical nano-metrology [0.0]
  reconstruction is a common problem in optical nano metrology. We present a Bayesian Target Vector Optimization scheme which combines two approaches. We find that the presented method generally uses fewer calls of the model function than any of the competing schemes to achieve similar reconstruction performance.
  arXiv  Detail & Related papers  (2021-07-12T15:32:15Z)
• Gaussian Process Latent Class Choice Models [7.992550355579791]
  We present a non-parametric class of probabilistic machine learning within discrete choice models (DCMs) The proposed model would assign individuals probabilistically to behaviorally homogeneous clusters (latent classes) using GPs. The model is tested on two different mode choice applications and compared against different LCCM benchmarks.
  arXiv  Detail & Related papers  (2021-01-28T19:56:42Z)
• Community Detection in the Stochastic Block Model by Mixed Integer Programming [3.8073142980733]
  Degree-Corrected Block Model (DCSBM) is a popular model to generate random graphs with community structure given an expected degree sequence. Standard approach of community detection based on the DCSBM is to search for the model parameters that are the most likely to have produced the observed network data through maximum likelihood estimation (MLE) We present mathematical programming formulations and exact solution methods that can provably find the model parameters and community assignments of maximum likelihood given an observed graph.
  arXiv  Detail & Related papers  (2021-01-26T22:04:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.