LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation

• URL: http://arxiv.org/abs/2605.22054v1
• Date: Thu, 21 May 2026 06:42:47 GMT
• Title: LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation
• Abstract summary: Large language models (LLMs) are knowledge-driven components in Bayesian optimization.<n>We propose a framework that combines LLM predictions with experimental observations within a single BO loop.<n>We show that LABO consistently outperforms existing methods under identical experimental budgets.
• Score: 18.30782195527165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating criterion to dynamically balance the reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.

Related papers

• Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery [32.51891966529322]
  We propose LLM-Guided Bayesian Optimization (LGBO), a preference-guided BO framework that integrates semantic reasoning into the optimization loop.<n>In a new wet-lab optimization of Fe-Cr battery electrolytes, LGBO attains textbf90% of the best observed value within 6, whereas standard BO and existing LLM-augmented baselines require more than 10 iterations.
  arXiv  Detail & Related papers  (2026-05-18T07:30:25Z)
• BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks [42.78958516953461]
  Black-box optimization methods offer a promising yet underexplored direction for principled, sample-efficient methods.<n>New methods are often only evaluated on synthetic test functions and small-scale datasets.<n>We introduce BoLT, the first LLM-centric benchmark that democratizes research for the BBO community.
  arXiv  Detail & Related papers  (2026-05-16T13:53:44Z)
• LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? [25.116042999105193]
  Large language models (LLMs) have been proposed as general-purpose agents for experimental design.<n>We evaluate this hypothesis using both open- and closed-source instruction-tuned LLMs applied to genetic perturbation and molecular property discovery tasks.<n>We find that LLM-based agents show no sensitivity to experimental feedback, replacing true outcomes with randomly permuted labels has no impact on performance.
  arXiv  Detail & Related papers  (2025-09-24T15:50:17Z)
• ChemBOMAS: Accelerated BO in Chemistry with LLM-Enhanced Multi-Agent System [72.63341091857959]
  We introduce ChemBOMAS: a large language model (LLM)-enhanced multi-agent system that accelerates Bayesian optimization.<n>Data-driven strategy involves an 8B-scale LLM regressor fine-tuned on a mere 1% labeled samples.<n>The knowledge-driven strategy employs a hybrid Retrieval-Augmented Generation approach to guide LLM in dividing the search space.<n>ChemBOMAS set a new state-of-the-art, accelerating optimization efficiency by up to 5-fold compared to baseline methods.
  arXiv  Detail & Related papers  (2025-09-10T16:24:08Z)
• MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search [102.11776494401705]
  Large language models (LLMs) have shown promise in automating scientific hypothesis generation.<n>Existing approaches primarily yield coarse-grained hypotheses lacking critical methodological and experimental details.<n>We introduce and formally define the new task of fine-grained scientific hypothesis discovery.
  arXiv  Detail & Related papers  (2025-05-25T16:13:46Z)
• LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization [5.844783557050259]
  Large Language Models (LLMs) have shown remarkable adaptability in low-data regimes.<n>We propose LLINBO: LLM-in-the-Loop BO, a hybrid framework for BO that combines LLMs with statistical surrogate experts.<n>We end the paper with a real-life proof-of-concept in the context of 3D printing.
  arXiv  Detail & Related papers  (2025-05-20T15:54:48Z)
• Sequential Large Language Model-Based Hyper-parameter Optimization [0.0]
  This study introduces SLLMBO, an innovative framework leveraging large language models (LLMs) for hyper- parameter optimization (HPO)<n>It incorporates dynamic search space adaptability, enhanced parameter space exploitation, and a novel LLM-tree-structured parzen estimator (LLM-TPE) sampler.<n>This comprehensive benchmarking evaluates multiple LLMs, including GPT-3.5-Turbo, GPT-4o, Claude-Sonnet-3.5, and Gemini-1.5-Flash.
  arXiv  Detail & Related papers  (2024-10-27T00:50:30Z)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty.<n>We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications.<n>Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• Large Language Models to Enhance Bayesian Optimization [57.474613739645605]
  We present LLAMBO, a novel approach that integrates the capabilities of Large Language Models (LLM) within Bayesian optimization. At a high level, we frame the BO problem in natural language, enabling LLMs to iteratively propose and evaluate promising solutions conditioned on historical evaluations. Our findings illustrate that LLAMBO is effective at zero-shot warmstarting, and enhances surrogate modeling and candidate sampling, especially in the early stages of search when observations are sparse.
  arXiv  Detail & Related papers  (2024-02-06T11:44:06Z)
• On Learning to Summarize with Large Language Models as References [101.79795027550959]
  Large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets. We study an LLM-as-reference learning setting for smaller text summarization models to investigate whether their performance can be substantially improved.
  arXiv  Detail & Related papers  (2023-05-23T16:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.