Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions

• URL: http://arxiv.org/abs/2504.08874v1
• Date: Fri, 11 Apr 2025 12:45:07 GMT
• Title: Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions
• Authors: Roshan Patel, Saeed Moayedpour, Louis De Lescure, Lorenzo Kogler-Anele, Alan Cherney, Sven Jager, Yasser Jangjou,
• Abstract summary: Large language models (LLMs) have demonstrated that chemical information present in foundation training data can give them utility for processing chemical data.<n>We show that chemical information from LLMs can be elicited and used for transfer learning to accelerate the BO of reaction conditions to maximize yield.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Machine learning and Bayesian optimization (BO) algorithms can significantly accelerate the optimization of chemical reactions. Transfer learning can bolster the effectiveness of BO algorithms in low-data regimes by leveraging pre-existing chemical information or data outside the direct optimization task (i.e., source data). Large language models (LLMs) have demonstrated that chemical information present in foundation training data can give them utility for processing chemical data. Furthermore, they can be augmented with and help synthesize potentially multiple modalities of source chemical data germane to the optimization task. In this work, we examine how chemical information from LLMs can be elicited and used for transfer learning to accelerate the BO of reaction conditions to maximize yield. Specifically, we show that a survey-like prompting scheme and preference learning can be used to infer a utility function which models prior chemical information embedded in LLMs over a chemical parameter space; we find that the utility function shows modest correlation to true experimental measurements (yield) over the parameter space despite operating in a zero-shot setting. Furthermore, we show that the utility function can be leveraged to focus BO efforts in promising regions of the parameter space, improving the yield of the initial BO query and enhancing optimization in 4 of the 6 datasets studied. Overall, we view this work as a step towards bridging the gap between the chemistry knowledge embedded in LLMs and the capabilities of principled BO methods to accelerate reaction optimization.

Related papers

• ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning [64.2106664137118]
  ChemAgent is a novel framework designed to improve the performance of large language models (LLMs)<n>It is developed by decomposing chemical tasks into sub-tasks and compiling these sub-tasks into a structured collection that can be referenced for future queries.<n>When presented with a new problem, ChemAgent retrieves and refines pertinent information from the library, which we call memory.
  arXiv  Detail & Related papers  (2025-01-11T17:10:30Z)
• Text-Guided Multi-Property Molecular Optimization with a Diffusion Language Model [77.50732023411811]
  We propose a text-guided multi-property molecular optimization method utilizing transformer-based diffusion language model (TransDLM) TransDLM leverages standardized chemical nomenclature as semantic representations of molecules and implicitly embeds property requirements into textual descriptions. Our approach surpasses state-of-the-art methods in optimizing molecular structural similarity and enhancing chemical properties on the benchmark dataset.
  arXiv  Detail & Related papers  (2024-10-17T14:30:27Z)
• Ranking over Regression for Bayesian Optimization and Molecule Selection [0.0680892187976602]
  We introduce Rank-based Bayesian Optimization (RBO), which utilizes a ranking model as the surrogate. We present a comprehensive investigation of RBO's optimization performance compared to conventional BO on various chemical datasets. We conclude RBO is an effective alternative to regression-based BO, especially for optimizing novel chemical compounds.
  arXiv  Detail & Related papers  (2024-10-11T22:38:14Z)
• Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation [11.778576032848482]
  This work enhances such models by improving their inference and evaluation capabilities with minimal or no additional training. We reveal intriguing insights into the behaviour and suitability of such methods while significantly surpassing state-of-the-art models. We propose a novel atomic-level evaluation method leveraging off-the-shelf Natural Language Inference (NLI) models for use in the unseen chemical domain.
  arXiv  Detail & Related papers  (2024-05-22T20:40:53Z)
• An Autonomous Large Language Model Agent for Chemical Literature Data Mining [60.85177362167166]
  We introduce an end-to-end AI agent framework capable of high-fidelity extraction from extensive chemical literature. Our framework's efficacy is evaluated using accuracy, recall, and F1 score of reaction condition data.
  arXiv  Detail & Related papers  (2024-02-20T13:21:46Z)
• Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
  This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning. Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques. Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
  arXiv  Detail & Related papers  (2024-02-18T14:08:48Z)
• DrugAssist: A Large Language Model for Molecule Optimization [29.95488215594247]
  DrugAssist is an interactive molecule optimization model that performs optimization through human-machine dialogue. DrugAssist has achieved leading results in both single and multiple property optimization. We publicly release a large instruction-based dataset called MolOpt-Instructions for fine-tuning language models on molecule optimization tasks.
  arXiv  Detail & Related papers  (2023-12-28T10:46:56Z)
• Learning Implicit Priors for Motion Optimization [105.11889448885226]
  Energy-based Models (EBM) represent expressive probability density distributions. We present a set of required modeling and algorithmic choices to adapt EBMs into motion optimization.
  arXiv  Detail & Related papers  (2022-04-11T19:14:54Z)
• Improving Molecular Representation Learning with Metric Learning-enhanced Optimal Transport [49.237577649802034]
  We develop a novel optimal transport-based algorithm termed MROT to enhance their generalization capability for molecular regression problems. MROT significantly outperforms state-of-the-art models, showing promising potential in accelerating the discovery of new substances.
  arXiv  Detail & Related papers  (2022-02-13T04:56:18Z)
• Scalable and Flexible Deep Bayesian Optimization with Auxiliary Information for Scientific Problems [10.638330155988145]
  We propose performing Bayesian optimization on complex, structured problems by using Bayesian Neural Networks (BNNs) BNNs have the representation power and flexibility to handle structured data and exploit auxiliary information. We show that BNNs often outperform GPs as surrogate models for BO in terms of both sampling efficiency and computational cost.
  arXiv  Detail & Related papers  (2021-04-23T15:46:37Z)
• CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method [2.8381402107366034]
  We propose a combined machine learning and molecular modeling approach that automates lead optimization workflow. Our method provides new hints for drug modification hotspots which can be used to improve drug efficacy.
  arXiv  Detail & Related papers  (2020-11-27T15:41:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.