Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing

• URL: http://arxiv.org/abs/2212.10789v3
• Date: Mon, 29 Jan 2024 22:09:49 GMT
• Title: Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing
• Authors: Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, Anima Anandkumar
• Abstract summary: We present a multi-modal molecule structure-text model, MoleculeSTM, by jointly learning molecules' chemical structures and textual descriptions. In experiments, MoleculeSTM obtains the state-of-the-art generalization ability to novel biochemical concepts.
• Score: 107.49804059269212
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There is increasing adoption of artificial intelligence in drug discovery. However, existing studies use machine learning to mainly utilize the chemical structures of molecules but ignore the vast textual knowledge available in chemistry. Incorporating textual knowledge enables us to realize new drug design objectives, adapt to text-based instructions and predict complex biological activities. Here we present a multi-modal molecule structure-text model, MoleculeSTM, by jointly learning molecules' chemical structures and textual descriptions via a contrastive learning strategy. To train MoleculeSTM, we construct a large multi-modal dataset, namely, PubChemSTM, with over 280,000 chemical structure-text pairs. To demonstrate the effectiveness and utility of MoleculeSTM, we design two challenging zero-shot tasks based on text instructions, including structure-text retrieval and molecule editing. MoleculeSTM has two main properties: open vocabulary and compositionality via natural language. In experiments, MoleculeSTM obtains the state-of-the-art generalization ability to novel biochemical concepts across various benchmarks.

Related papers

• MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild [23.78185449646608]
  We present Mol, a novel end-to-end optical chemical structure recognition method. We use a SMILES encoding rule to annotate Mol-7M, the largest annotated molecular image dataset. We trained an end-to-end molecular image captioning model, Mol, using a curriculum learning approach.
  arXiv  Detail & Related papers  (2024-11-17T15:00:09Z)
• UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation [35.51027934845928]
  We introduce UniMoT, a Unified Molecule-Text LLM adopting a tokenizer-based architecture. A Vector Quantization-driven tokenizer transforms molecules into sequences of molecule tokens with causal dependency. UniMoT emerges as a multi-modal generalist capable of performing both molecule-to-text and text-to-molecule tasks.
  arXiv  Detail & Related papers  (2024-08-01T18:31:31Z)
• Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models [43.26037039251725]
  We propose AMOLE, which augments molecule-text pairs with structural similarity preserving loss. We also propose an expertise reconstruction loss to transfer knowledge from molecules that have extensive expertise to those with less expertise.
  arXiv  Detail & Related papers  (2024-07-12T07:09:10Z)
• Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Model [49.64512917330373]
  We introduce a multi-constraint molecular generation large language model, TSMMG, akin to a student. To train TSMMG, we construct a large set of text-molecule pairs by extracting molecular knowledge from these 'teachers' We experimentally show that TSMMG remarkably performs in generating molecules meeting complex, natural language-described property requirements.
  arXiv  Detail & Related papers  (2024-03-20T02:15:55Z)
• Bridging Text and Molecule: A Survey on Multimodal Frameworks for Molecule [16.641797535842752]
  In this paper, we present the first systematic survey on multimodal frameworks for molecules research. We begin with the development of molecular deep learning and point out the necessity to involve textual modality. Furthermore, we delves into the utilization of large language models and prompting techniques for molecular tasks and present significant applications in drug discovery.
  arXiv  Detail & Related papers  (2024-03-07T03:03:13Z)
• Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey [75.47055414002571]
  The integration of biomolecular modeling with natural language (BL) has emerged as a promising interdisciplinary area at the intersection of artificial intelligence, chemistry and biology. We provide an analysis of recent advancements achieved through cross modeling of biomolecules and natural language.
  arXiv  Detail & Related papers  (2024-03-03T14:59:47Z)
• Interactive Molecular Discovery with Natural Language [69.89287960545903]
  We propose the conversational molecular design, a novel task adopting natural language for describing and editing target molecules. To better accomplish this task, we design ChatMol, a knowledgeable and versatile generative pre-trained model, enhanced by injecting experimental property information.
  arXiv  Detail & Related papers  (2023-06-21T02:05:48Z)
• MolXPT: Wrapping Molecules with Text for Generative Pre-training [141.0924452870112]
  MolXPT is a unified language model of text and molecules pre-trained on SMILES wrapped by text. MolXPT outperforms strong baselines of molecular property prediction on MoleculeNet.
  arXiv  Detail & Related papers  (2023-05-18T03:58:19Z)
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [63.60376252491507]
  We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
  arXiv  Detail & Related papers  (2022-09-12T00:56:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.