Benchmarking Generative AI Against Bayesian Optimization for Constrained Multi-Objective Inverse Design

• URL: http://arxiv.org/abs/2511.00070v1
• Date: Wed, 29 Oct 2025 10:37:09 GMT
• Title: Benchmarking Generative AI Against Bayesian Optimization for Constrained Multi-Objective Inverse Design
• Authors: Muhammad Bilal Awan, Abdul Razzaq, Abdul Shahid,
• Abstract summary: This paper investigates the performance of Large Language Models (LLMs) as generative feasibles for solving constrained multi-objective regression tasks.<n>The best-performing LLM (Math-7B) achieved a Generational Distance (GD) of 1.21, significantly outperforming the traditional BoTorch Ax baseline.<n>The findings have direct industrial applications in optimizing formulation design for resins, rheological, and chemical properties.
• Score: 0.15293427903448018
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates the performance of Large Language Models (LLMs) as generative optimizers for solving constrained multi-objective regression tasks, specifically within the challenging domain of inverse design (property-to-structure mapping). This problem, critical to materials informatics, demands finding complex, feasible input vectors that lie on the Pareto optimal front. While LLMs have demonstrated universal effectiveness across generative and reasoning tasks, their utility in constrained, continuous, high-dimensional numerical spaces tasks they weren't explicitly architected for remains an open research question. We conducted a rigorous comparative study between established Bayesian Optimization (BO) frameworks and a suite of fine-tuned LLMs and BERT models. For BO, we benchmarked the foundational BoTorch Ax implementation against the state-of-the-art q-Expected Hypervolume Improvement (qEHVI, BoTorchM). The generative approach involved fine-tuning models via Parameter-Efficient Fine-Tuning (PEFT), framing the challenge as a regression problem with a custom output head. Our results show that BoTorch qEHVI achieved perfect convergence (GD=0.0), setting the performance ceiling. Crucially, the best-performing LLM (WizardMath-7B) achieved a Generational Distance (GD) of 1.21, significantly outperforming the traditional BoTorch Ax baseline (GD=15.03). We conclude that specialized BO frameworks remain the performance leader for guaranteed convergence, but fine-tuned LLMs are validated as a promising, computationally fast alternative, contributing essential comparative metrics to the field of AI-driven optimization. The findings have direct industrial applications in optimizing formulation design for resins, polymers, and paints, where multi-objective trade-offs between mechanical, rheological, and chemical properties are critical to innovation and production efficiency.

Related papers

• Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning [17.84264663466905]
  Surrogate-assisted Evolutionary Algorithms(SAEAs) have shown promising robustness in solving expensive optimization problems.<n>A key aspect that impacts SAEAs' effectiveness is surrogate model selection, which in existing works is predominantly decided by human developer.<n>We propose a reinforcement learning-assisted ensemble framework, termed as SEEMOO, which is capable of scheduling different surrogate models within a single optimization process.
  arXiv  Detail & Related papers  (2026-01-31T06:14:27Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancement [5.852607388888843]
  Large Language Model (LLM)-based agents demonstrate advanced reasoning capabilities, yet practical constraints frequently limit outputs to single responses.<n>This paper introduces MARINE, a framework that reconceptualizes test-time reasoning as iterative refinement of a persistent reference trajectory.<n>Proposed MARINE delivers higher-quality samples to alignment and optimization processes than traditional sampling-and-ranking strategies.
  arXiv  Detail & Related papers  (2025-12-05T11:19:18Z)
• Relation-Aware Bayesian Optimization of DBMS Configurations Guided by Affinity Scores [2.474203056060563]
  Database Management Systems (DBMSs) are fundamental for managing large-scale and heterogeneous data, and their performance is critically influenced by configuration parameters.<n>Recent research has focused on automated configuration optimization using machine learning; however, existing approaches still exhibit several key limitations.<n>We propose RelTune, a novel framework that represents parameter dependencies as a Graph and learns GNN-based latent embeddings that encode performancerelevant semantics.
  arXiv  Detail & Related papers  (2025-10-31T03:46:42Z)
• MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization [103.74675519953898]
  Long-chain reflective reasoning is a prerequisite for solving complex real-world problems.<n>We build a benchmark consisting 1,260 samples of 42 challenging synthetic tasks.<n>We generate post-training data and explore learning paradigms for exploiting such data.
  arXiv  Detail & Related papers  (2025-10-09T17:53:58Z)
• Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimization [4.12346015436419]
  Black-box optimization problems with high evaluation costs entail solving black-box optimization problems with sample efficiency.<n>We propose a neural network-based surrogate to model the mapping from the high-dimensional input variables to the objective function.<n>We demonstrate the compelling performance of the proposed (ENS-)LoRA-VBLL approaches on various high-dimensional benchmarks and the real-world molecular optimization tasks.
  arXiv  Detail & Related papers  (2025-10-01T21:28:50Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• Multi-fidelity Bayesian Data-Driven Design of Energy Absorbing Spinodoid Cellular Structures [1.5187678526255866]
  This paper aims to address shortcomings by employing Sobol' samples with variance-based sensitivity analysis.<n>The results serve to support the utility of multi-fidelity techniques across expensive data-driven design problems.
  arXiv  Detail & Related papers  (2025-07-25T15:55:59Z)
• ROMO: Retrieval-enhanced Offline Model-based Optimization [14.277672372460785]
  Data-driven black-box model-based optimization (MBO) problems arise in a number of practical application scenarios. We propose retrieval-enhanced offline model-based optimization (ROMO) ROMO is simple to implement and outperforms state-of-the-art approaches in the CoMBO setting.
  arXiv  Detail & Related papers  (2023-10-11T15:04:33Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Machine Learning Framework for Quantum Sampling of Highly-Constrained, Continuous Optimization Problems [101.18253437732933]
  We develop a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate unconstrained binary optimization problems. We showcase the framework's performance on two inverse design problems by optimizing thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering.
  arXiv  Detail & Related papers  (2021-05-06T02:22:23Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.