Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

• URL: http://arxiv.org/abs/2607.29225v1
• Date: Fri, 31 Jul 2026 09:55:38 GMT
• Title: Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery
• Abstract summary: We evaluate four surrogate models: Gaussian Process, Random Forests, NGBoost, and Bayesian Adaptive Spline Surfaces.<n>We show that Gaussian Process-based BO incurs the highest time and memory overhead without delivering superior optimization or sample efficiency.<n>We introduce a surrogate-recommendation framework that predicts the most suitable BO surrogate from inexpensive dataset characteristics.
• Score: 0.6927055673104934
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance. Here we present a systematic, compute-aware study of BO that evaluates surrogate models along two axes: optimization quality and computational frugality. Across eight benchmark functions and nine real-world datasets spanning materials science, mechanics, robotics, chemistry, and machine learning, we benchmark four surrogate models: Gaussian Processes, Random Forests, NGBoost, and Bayesian Adaptive Spline Surfaces. We show that Gaussian Process-based BO consistently incurs the highest time and memory overhead without delivering superior optimization or sample efficiency. In contrast, scalable alternatives achieve equal or better performance at a fraction of the computational cost. Motivated by these findings, we introduce a surrogate-recommendation framework that predicts the most suitable BO surrogate from inexpensive dataset characteristics. Together, these results establish FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provide practical guidance for surrogate selection under limited computational and experimental budgets.

Related papers

• Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search [51.33002870475756]
  We introduce Power-Law Entropy Search (PLES), a cost-aware acquisition function built on multi-fidelity Bayesian optimization.<n>PLES searches for candidates that reduce the overall uncertainty of a scaling law estimate, instead of optimizing a single objective function.
  arXiv  Detail & Related papers  (2026-09-01T15:41:59Z)
• Modernizing HEBO: a robust Bayesian optimization baseline for practical heteroskedastic and non-stationary problems [0.0]
  tidyHEBO is a robust Bayesian optimization model inspired by heteroskedastic evolutionary Bayesian optimization (HEBO)<n> tidyHEBO reconstructs the HEBO design philosophy in BoTorch and revises surrogate training, output-warping selection and acquisition function evaluation.<n>We benchmarked tidyHEBO on synthetic functions, fully experimental reaction-optimization datasets and needle-in-a-haystack (NIAH) materials problems.
  arXiv  Detail & Related papers  (2026-07-12T09:27:57Z)
• Autocorrelated Optimize-via-Estimate: Predict-then-Optimize versus Finite-sample Optimal [2.0228793142608588]
  Models that directly optimize for out-of-sample performance in the finite-sample regime have emerged as a promising alternative to traditional estimate-then-optimize approaches.<n>We compare their performance in the context of autocorrelated uncertainties, specifically, under a Vector Autoregressive Moving Average VARMA(p,q) process.
  arXiv  Detail & Related papers  (2026-02-02T09:49:51Z)
• Adaptive Bayesian Data-Driven Design of Reliable Solder Joints for Micro-electronic Devices [0.0]
  Solder joint reliability related to failures due to thermomechanical loading is a critically important yet physically complex engineering problem.<n>In an increasingly data-driven world, the usage of efficient data-driven design schemes is a popular choice.<n>The authors argue that computational savings can be obtained from exploiting thorough surrogate modeling.
  arXiv  Detail & Related papers  (2025-07-25T20:34:03Z)
• Gradient-based Sample Selection for Faster Bayesian Optimization [10.137754677594861]
  We propose a novel approach, gradient-based sample selection Bayesian Optimization (GSSBO), to enhance the computational efficiency of BO.<n>Our approach significantly reduces the computational cost of GP fitting in BO while maintaining optimization performance comparable to baseline methods.
  arXiv  Detail & Related papers  (2025-04-10T13:38:15Z)
• Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing [53.77822620185878]
  We propose a synergistic methodology to concurrently optimize perovskite memristor fabrication and develop robust analog DNNs.<n>We develop "BayesMulti", a training strategy utilizing BO-guided noise injection to improve the resistance of analog DNNs to memristor imperfections.<n>Our integrated approach enables use of analog computing in much deeper and wider networks, achieving up to 100-fold improvements.
  arXiv  Detail & Related papers  (2024-12-03T19:20:08Z)
• Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models [79.84205827056907]
  We present Self-Steering Optimization ($SSO$), an algorithm that autonomously generates high-quality preference data.<n>$SSO$ employs a specialized optimization objective to build a data generator from the policy model itself, which is used to produce accurate and on-policy data.<n>Our evaluation shows that $SSO$ consistently outperforms baselines in human preference alignment and reward optimization.
  arXiv  Detail & Related papers  (2024-10-22T16:04:03Z)
• Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation [55.75188191403343]
  We introduce utility, which is a function predefined by each user and describes the trade-off between cost and performance of BO. We validate our algorithm on various LC datasets and found it outperform all the previous multi-fidelity BO and transfer-BO baselines we consider.
  arXiv  Detail & Related papers  (2024-05-28T07:38:39Z)
• Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties [49.351577714596544]
  We propose a human-AI collaborative Bayesian framework to incorporate expert preferences about unmeasured abstract properties into surrogate modeling. We provide an efficient strategy that can also handle any incorrect/misleading expert bias in preferential judgments.
  arXiv  Detail & Related papers  (2024-02-27T09:23:13Z)
• Sample-Efficient and Surrogate-Based Design Optimization of Underwater Vehicle Hulls [0.4543820534430522]
  We show that theBO-LCB algorithm is the most sample-efficient optimization framework and has the best convergence behavior of those considered. We also show that our DNN-based surrogate model predicts drag force on test data in tight agreement with CFD simulations, with a mean absolute percentage error (MAPE) of 1.85%. We demonstrate a two-orders-of-magnitude speedup for the design optimization process when the surrogate model is used.
  arXiv  Detail & Related papers  (2023-04-24T19:52:42Z)
• Global Optimization of Gaussian processes [52.77024349608834]
  We propose a reduced-space formulation with trained Gaussian processes trained on few data points. The approach also leads to significantly smaller and computationally cheaper sub solver for lower bounding. In total, we reduce time convergence by orders of orders of the proposed method.
  arXiv  Detail & Related papers  (2020-05-21T20:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.