Benchmarking Generative AI Against Bayesian Optimization for Constrained Multi-Objective Inverse Design

• URL: http://arxiv.org/abs/2511.00070v1
• Date: Wed, 29 Oct 2025 10:37:09 GMT
• Title: Benchmarking Generative AI Against Bayesian Optimization for Constrained Multi-Objective Inverse Design
• Abstract summary: This paper investigates the performance of Large Language Models (LLMs) as generative feasibles for solving constrained multi-objective regression tasks.<n>The best-performing LLM (Math-7B) achieved a Generational Distance (GD) of 1.21, significantly outperforming the traditional BoTorch Ax baseline.<n>The findings have direct industrial applications in optimizing formulation design for resins, rheological, and chemical properties.
• Score: 0.15293427903448018
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates the performance of Large Language Models (LLMs) as generative optimizers for solving constrained multi-objective regression tasks, specifically within the challenging domain of inverse design (property-to-structure mapping). This problem, critical to materials informatics, demands finding complex, feasible input vectors that lie on the Pareto optimal front. While LLMs have demonstrated universal effectiveness across generative and reasoning tasks, their utility in constrained, continuous, high-dimensional numerical spaces tasks they weren't explicitly architected for remains an open research question. We conducted a rigorous comparative study between established Bayesian Optimization (BO) frameworks and a suite of fine-tuned LLMs and BERT models. For BO, we benchmarked the foundational BoTorch Ax implementation against the state-of-the-art q-Expected Hypervolume Improvement (qEHVI, BoTorchM). The generative approach involved fine-tuning models via Parameter-Efficient Fine-Tuning (PEFT), framing the challenge as a regression problem with a custom output head. Our results show that BoTorch qEHVI achieved perfect convergence (GD=0.0), setting the performance ceiling. Crucially, the best-performing LLM (WizardMath-7B) achieved a Generational Distance (GD) of 1.21, significantly outperforming the traditional BoTorch Ax baseline (GD=15.03). We conclude that specialized BO frameworks remain the performance leader for guaranteed convergence, but fine-tuned LLMs are validated as a promising, computationally fast alternative, contributing essential comparative metrics to the field of AI-driven optimization. The findings have direct industrial applications in optimizing formulation design for resins, polymers, and paints, where multi-objective trade-offs between mechanical, rheological, and chemical properties are critical to innovation and production efficiency.

Related papers

• A FEM-Based Surrogate Modelling and Optimization Framework for Physics-Constrained Electromagnetic Coil Design [2.6205603301997855]
  A Python--MPh--COMSOL workflow couples a two-dimensional axisymmetric finite-element method (FEM) to a Matern 5/2 Gaussian-process (GP) probabilistic surrogate.<n>A selected-design check at a common total current preserves the observed BOBYQA--COBYLA--EI-BO ordering.
  arXiv  Detail & Related papers  (2026-08-19T13:27:15Z)
• Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization [85.4863852505905]
  We introduce unsupervised reward optimization of protein language models (PLMs)<n>Our key insight is that task-agnostic rewards, which combine intrinsic model uncertainty with semantic consistency informed by protein representation models, exhibit strong correlation with controllability measures across base models and temperature regimes.<n>Our framework provides a scalable pathway toward controllable biomolecular design in settings where labeled preferences or experimental feedback are scarce or unavailable.
  arXiv  Detail & Related papers  (2026-06-17T11:42:01Z)
• The Illusion of Multi-Agent Advantage [63.92466143571383]
  We show that expert-architected Multi-Agent Systems (MAS) consistently outperforms automatically generated architectures in raw performance and cost-efficiency.<n>Existing evaluation frameworks mask critical architectural gaps and inefficiencies by failing to account for the marginal utility of increased computational cost.
  arXiv  Detail & Related papers  (2026-06-11T07:39:24Z)
• FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization [61.43300970020897]
  Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation.<n>Existing benchmarks are limited to small or simplified examples far below real-world scale and complexity.<n>We introduce FrontierOR, among the first benchmarks to evaluate LLM-based efficient algorithm design for realistic large-scale optimization problems.
  arXiv  Detail & Related papers  (2026-05-24T20:10:42Z)
• Support-Proximity Augmented Diffusion Estimation for Offline Black-Box Optimization [17.43522044331905]
  We propose SPADE (Support-Proximity Augmented Diffusion Estimation), a novel framework that reimagines forward surrogate modeling through the lens of conditional generative modeling.<n>We show that SPADE achieves state-of-the-art performance across Design-Bench tasks and an LLM data mixture optimization benchmark.
  arXiv  Detail & Related papers  (2026-05-11T21:09:28Z)
• Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning [17.84264663466905]
  Surrogate-assisted Evolutionary Algorithms(SAEAs) have shown promising robustness in solving expensive optimization problems.<n>A key aspect that impacts SAEAs' effectiveness is surrogate model selection, which in existing works is predominantly decided by human developer.<n>We propose a reinforcement learning-assisted ensemble framework, termed as SEEMOO, which is capable of scheduling different surrogate models within a single optimization process.
  arXiv  Detail & Related papers  (2026-01-31T06:14:27Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancement [5.852607388888843]
  Large Language Model (LLM)-based agents demonstrate advanced reasoning capabilities, yet practical constraints frequently limit outputs to single responses.<n>This paper introduces MARINE, a framework that reconceptualizes test-time reasoning as iterative refinement of a persistent reference trajectory.<n>Proposed MARINE delivers higher-quality samples to alignment and optimization processes than traditional sampling-and-ranking strategies.
  arXiv  Detail & Related papers  (2025-12-05T11:19:18Z)
• Relation-Aware Bayesian Optimization of DBMS Configurations Guided by Affinity Scores [2.474203056060563]
  Database Management Systems (DBMSs) are fundamental for managing large-scale and heterogeneous data, and their performance is critically influenced by configuration parameters.<n>Recent research has focused on automated configuration optimization using machine learning; however, existing approaches still exhibit several key limitations.<n>We propose RelTune, a novel framework that represents parameter dependencies as a Graph and learns GNN-based latent embeddings that encode performancerelevant semantics.
  arXiv  Detail & Related papers  (2025-10-31T03:46:42Z)
• MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization [103.74675519953898]
  Long-chain reflective reasoning is a prerequisite for solving complex real-world problems.<n>We build a benchmark consisting 1,260 samples of 42 challenging synthetic tasks.<n>We generate post-training data and explore learning paradigms for exploiting such data.
  arXiv  Detail & Related papers  (2025-10-09T17:53:58Z)
• Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimization [4.12346015436419]
  Black-box optimization problems with high evaluation costs entail solving black-box optimization problems with sample efficiency.<n>We propose a neural network-based surrogate to model the mapping from the high-dimensional input variables to the objective function.<n>We demonstrate the compelling performance of the proposed (ENS-)LoRA-VBLL approaches on various high-dimensional benchmarks and the real-world molecular optimization tasks.
  arXiv  Detail & Related papers  (2025-10-01T21:28:50Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• Multi-fidelity Bayesian Data-Driven Design of Energy Absorbing Spinodoid Cellular Structures [1.5187678526255866]
  This paper aims to address shortcomings by employing Sobol' samples with variance-based sensitivity analysis.<n>The results serve to support the utility of multi-fidelity techniques across expensive data-driven design problems.
  arXiv  Detail & Related papers  (2025-07-25T15:55:59Z)
• ROMO: Retrieval-enhanced Offline Model-based Optimization [14.277672372460785]
  Data-driven black-box model-based optimization (MBO) problems arise in a number of practical application scenarios. We propose retrieval-enhanced offline model-based optimization (ROMO) ROMO is simple to implement and outperforms state-of-the-art approaches in the CoMBO setting.
  arXiv  Detail & Related papers  (2023-10-11T15:04:33Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Machine Learning Framework for Quantum Sampling of Highly-Constrained, Continuous Optimization Problems [101.18253437732933]
  We develop a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate unconstrained binary optimization problems. We showcase the framework's performance on two inverse design problems by optimizing thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering.
  arXiv  Detail & Related papers  (2021-05-06T02:22:23Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.