PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes

• URL: http://arxiv.org/abs/2406.13193v1
• Date: Wed, 19 Jun 2024 03:59:46 GMT
• Title: PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes
• Authors: He Cao, Yanjun Shao, Zhiyuan Liu, Zijing Liu, Xiangru Tang, Yuan Yao, Yu Li,
• Abstract summary: Multimodal Large Language Models (MLLMs) have seen growing adoption across various scientific disciplines. Current approaches, however, often neglect the critical role of multiple molecule graph interaction in understanding chemical reactions. This study introduces PRESTO, a new framework that bridges the molecule-text modality gap by integrating a comprehensive benchmark of pretraining strategies and dataset configurations.
• Score: 33.293741487835824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) have seen growing adoption across various scientific disciplines. These advancements encourage the investigation of molecule-text modeling within synthetic chemistry, a field dedicated to designing and conducting chemical reactions to synthesize new compounds with desired properties and applications. Current approaches, however, often neglect the critical role of multiple molecule graph interaction in understanding chemical reactions, leading to suboptimal performance in synthetic chemistry tasks. This study introduces PRESTO(Progressive Pretraining Enhances Synthetic Chemistry Outcomes), a new framework that bridges the molecule-text modality gap by integrating a comprehensive benchmark of pretraining strategies and dataset configurations. It progressively improves multimodal LLMs through cross-modal alignment and multi-graph understanding. Our extensive experiments demonstrate that PRESTO offers competitive results in downstream synthetic chemistry tasks. The code can be found at https://github.com/IDEA-XL/PRESTO.

Related papers

• BatGPT-Chem: A Foundation Large Model For Retrosynthesis Prediction [65.93303145891628]
  BatGPT-Chem is a large language model with 15 billion parameters, tailored for enhanced retrosynthesis prediction. Our model captures a broad spectrum of chemical knowledge, enabling precise prediction of reaction conditions. This development empowers chemists to adeptly address novel compounds, potentially expediting the innovation cycle in drug manufacturing and materials science.
  arXiv  Detail & Related papers  (2024-08-19T05:17:40Z)
• ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area [50.15254966969718]
  We introduce textbfChemVLM, an open-source chemical multimodal large language model for chemical applications. ChemVLM is trained on a carefully curated bilingual dataset that enhances its ability to understand both textual and visual chemical information. We benchmark ChemVLM against a range of open-source and proprietary multimodal large language models on various tasks.
  arXiv  Detail & Related papers  (2024-08-14T01:16:40Z)
• ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining [76.51346919370005]
  We propose ReactXT for reaction-text modeling and OpenExp for experimental procedure prediction. ReactXT features three types of input contexts to incrementally pretrain LMs. Our code is available at https://github.com/syr-cn/ReactXT.
  arXiv  Detail & Related papers  (2024-05-23T06:55:59Z)
• ChemLLM: A Chemical Large Language Model [49.308528569982805]
  Large language models (LLMs) have made impressive progress in chemistry applications. However, the community lacks an LLM specifically designed for chemistry. Here, we introduce ChemLLM, a comprehensive framework that features the first LLM dedicated to chemistry.
  arXiv  Detail & Related papers  (2024-02-10T01:11:59Z)
• Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesis [57.70772230913099]
  Chemist-X automates the reaction condition recommendation (RCR) task in chemical synthesis with retrieval-augmented generation (RAG) technology. Chemist-X interrogates online molecular databases and distills critical data from the latest literature database. Chemist-X considerably reduces chemists' workload and allows them to focus on more fundamental and creative problems.
  arXiv  Detail & Related papers  (2023-11-16T01:21:33Z)
• Asymmetric Contrastive Multimodal Learning for Advancing Chemical Understanding [18.217304893881405]
  We introduce Asymmetric Contrastive Multimodal Learning (ACML) as a novel approach tailored for molecules. ACML harnesses the power of effective asymmetric contrastive learning to seamlessly transfer information from various chemical modalities to molecular graph representations. We demonstrate the effectiveness of this framework through large-scale cross-modality retrieval and isomer discrimination tasks.
  arXiv  Detail & Related papers  (2023-11-11T01:58:45Z)
• Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model [3.3031562864527664]
  We propose a unified framework that addresses both the reaction representation learning and molecule generation tasks. Inspired by the organic chemistry mechanism, we develop a novel pretraining framework that enables us to incorporate inductive biases into the model. Our framework achieves state-of-the-art results on challenging downstream tasks.
  arXiv  Detail & Related papers  (2023-03-13T10:06:41Z)
• Modern Hopfield Networks for Few- and Zero-Shot Reaction Prediction [3.885603826656419]
  Computer-assisted synthesis planning (CASP) to realize physical molecules is still in its infancy and lacks a performance level that would enable large-scale molecule discovery. We propose a novel reaction prediction approach that uses a deep learning architecture with modern Hopfield networks (MHNs) that is optimized by contrastive learning. We show that our MHN contrastive learning approach enables few- and zero-shot learning for reaction prediction which, in contrast to previous methods, can deal with rare, single, or even no training example(s) for a reaction.
  arXiv  Detail & Related papers  (2021-04-07T17:35:00Z)
• Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning [75.95376096628135]
  We propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space. We describe how the end-to-end training in this study represents an important paradigm in radically expanding the synthesizable chemical space.
  arXiv  Detail & Related papers  (2020-04-26T21:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.