LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms

• URL: http://arxiv.org/abs/2607.08791v1
• Date: Mon, 06 Jul 2026 08:44:13 GMT
• Title: LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms
• Abstract summary: We use large language models as mutation and crossover operators within evolutionary strategies to generate complete algorithm implementations.<n>Across nine evolutionary runs we generated approximately 900 algorithms and benchmarked them on twelve synthetic problems and three real-world engineering problems.
• Score: 2.012877764904354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Designing effective multi-objective Bayesian optimization (MOBO) algorithms requires balancing many interdependent design choices whose optimal configuration is problem-dependent and typically demands deep expertise. We extend the LLaMEA framework to MOBO, using large language models as mutation and crossover operators within evolutionary strategies to generate complete algorithm implementations, with SMAC hyperparameter optimization integrated into the evolutionary loop. Across nine evolutionary runs we generated approximately 900 algorithms and benchmarked them on twelve synthetic problems (ZDT, DTLZ, WFG) and three real-world engineering problems (RE), using a BoFire qParEGO implementation as a state-of-the-art Bayesian-optimization baseline. On the synthetic suite the strongest generated algorithm attains the highest mean normalized hypervolume (0.971, vs. 0.869 for qParEGO) while requiring roughly 60x less wall-clock time; a Friedman test with post-hoc analysis places the two in a single top-performing group, and per-problem tests find the generated algorithm significantly better than qParEGO on 7 of the 12 problems and never worse, matching state-of-the-art accuracy at an order-of-magnitude lower cost. On the three unseen real-world engineering problems a generated algorithm attains the best mean normalized hypervolume (0.985, vs. 0.971 for qParEGO)--significantly better than qParEGO on two of the three problems--at roughly 3.4x lower wall-clock cost, confirming that the gains transfer beyond the synthetic regime. LLM-driven evolutionary search can thus discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design.

Related papers

• A new dual-population constrained multi-objective evolutionary optimization algorithm with repair constraint handling for structural optimization [0.0]
  We introduce a novel constrained evolutionary optimization algorithm for multi-objective engineering applications.<n>The algorithm employs two interacting populations that exchange information, enabling effective global exploration and reducing the risk of convergence to local optima.<n>The proposed algorithm is tested on three engineering problems: the 72-bar truss, the 120-bar truss, and each involving hundreds of nonlinear failure constraints.
  arXiv  Detail & Related papers  (2026-07-14T00:57:07Z)
• FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization [61.43300970020897]
  Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation.<n>Existing benchmarks are limited to small or simplified examples far below real-world scale and complexity.<n>We introduce FrontierOR, among the first benchmarks to evaluate LLM-based efficient algorithm design for realistic large-scale optimization problems.
  arXiv  Detail & Related papers  (2026-05-24T20:10:42Z)
• optimize_anything: A Universal API for Optimizing any Text Parameter [98.42497715725356]
  We show a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs.<n>We show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm.
  arXiv  Detail & Related papers  (2026-05-19T10:18:12Z)
• S-LCG: Structured Linear Congruential Generator-Based Deterministic Algorithm for Search and Optimization [3.033196534183858]
  This study presents a novel deterministic optimization algorithm based on a special variant of the Linear Congruential Generator (LCG)<n>It is motivated by the intrinsic structure of the generator, the reason behind naming it the Structured Linear Congruential Generator (S- LCG)
  arXiv  Detail & Related papers  (2026-05-06T17:57:41Z)
• BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design [49.50107918295052]
  Large Language Model-based Hyper Heuristic (LHH) has recently emerged as an efficient way for automatic design.<n>We propose textbfBEAM (Bi-level Memory-adaptive Algorithmic Evolution) to address this problem.<n>We show that BEAM significantly outperforms existing LHHs, notably reducing the optimality gap by 37.84% on aggregate.
  arXiv  Detail & Related papers  (2026-04-14T15:46:47Z)
• K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model [57.440609834690385]
  Existing approaches treat Large Language Models (LLMs) as rapid code generators within evolutionary loops.<n>We propose Search via Co-Evolving World Model and build K-Search based on this method.<n>We evaluate K-Search on diverse, complex kernels FlashInfer, including GQA, MLA, and MoE kernels.
  arXiv  Detail & Related papers  (2026-02-22T11:06:22Z)
• QUASAR: An Evolutionary Algorithm to Accelerate High-Dimensional Optimization [0.0]
  This paper introduces Quasi-Adaptive Search with Asymptotic Reinitialization (QUASAR)<n>QUASAR is an evolutionary algorithm to accelerate convergence in complex, non-differentiable problems afflicted by the curse of dimensionality.
  arXiv  Detail & Related papers  (2025-11-17T19:02:31Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• A Novel Solver for QUBO Problems: Performance Analysis and Comparative Study with State-of-the-Art Algorithms [18.408755407920744]
  We introduce a quantum-inspired solver (QIS) that unites branch-and-bound pruning, continuous gradient-descent refinement, and quantum-inspired strategies.<n>We benchmark QIS3 against eight state-of-the-art solvers, including genetic algorithms, coherent Ising machines, simulated bifurcation.<n>Under a uniform runtime budget, QIS3 attains the best solution on nearly all instances, achieving optimality in 94% of max-cut instances.
  arXiv  Detail & Related papers  (2025-06-05T03:22:48Z)
• DAPO-QAOA: An algorithm for solving combinatorial optimization problems by dynamically constructing phase operators [3.146466735747661]
  We introduce a Dynamic Adaptive Phase Operator (DAPO) algorithm, which constructs phase operators based on the output of previous layers and neighborhood search approach.<n>DAPO achieves higher approximation ratios and significantly reduces two-qubit RZZ gates, especially in dense graphs.<n>Compared to vanilla QAOA, DAPO uses only 66% of RZZ gates at the same depth while delivering better results.
  arXiv  Detail & Related papers  (2025-02-06T14:24:05Z)
• Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling [96.47086913559289]
  gradient-based algorithms are widely used in bilevel optimization. We introduce a without-replacement sampling based algorithm which achieves a faster convergence rate. We validate our algorithms over both synthetic and real-world applications.
  arXiv  Detail & Related papers  (2024-11-07T17:05:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.