From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery

• URL: http://arxiv.org/abs/2309.05203v3
• Date: Tue, 5 Mar 2024 10:51:23 GMT
• Title: From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery
• Authors: Yuhan Chen, Nuwa Xi, Yanrui Du, Haochun Wang, Jianyu Chen, Sendong Zhao, Bing Qin
• Abstract summary: Cross-modal techniques for molecule discovery frequently encounter the issue of data scarcity, hampering their performance and application. We introduce a retrieval-based prompting strategy to construct high-quality pseudo data, then explore the optimal method to effectively leverage this pseudo data. Experiments show that using pseudo data for domain adaptation outperforms all existing methods, while also requiring a smaller model scale, reduced data size and lower training cost.
• Score: 35.5507452011217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Molecule discovery serves as a cornerstone in numerous scientific domains, fueling the development of new materials and innovative drug designs. Recent developments of in-silico molecule discovery have highlighted the promising results of cross-modal techniques, which bridge molecular structures with their descriptive annotations. However, these cross-modal methods frequently encounter the issue of data scarcity, hampering their performance and application. In this paper, we address the low-resource challenge by utilizing artificially-real data generated by Large Language Models (LLMs). We first introduce a retrieval-based prompting strategy to construct high-quality pseudo data, then explore the optimal method to effectively leverage this pseudo data. Experiments show that using pseudo data for domain adaptation outperforms all existing methods, while also requiring a smaller model scale, reduced data size and lower training cost, highlighting its efficiency. Furthermore, our method shows a sustained improvement as the volume of pseudo data increases, revealing the great potential of pseudo data in advancing low-resource cross-modal molecule discovery. Our code and data are available at https://github.com/SCIR-HI/ArtificiallyR2R.

Related papers

• KinDEL: DNA-Encoded Library Dataset for Kinase Inhibitors [2.0179908661487986]
  We present KinDEL, one of the first large, publicly available DEL datasets on two kinases. We benchmark different machine learning techniques to develop predictive models for hit identification. We provide biophysical assay data, both on- and off-DNA, to validate our models on a smaller subset of molecules.
  arXiv  Detail & Related papers  (2024-10-11T16:03:58Z)
• Extracting Training Data from Unconditional Diffusion Models [76.85077961718875]
  diffusion probabilistic models (DPMs) are being employed as mainstream models for generative artificial intelligence (AI) We aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization. Based on the theoretical analysis, we propose a novel data extraction method called textbfSurrogate condItional Data Extraction (SIDE) that leverages a trained on generated data as a surrogate condition to extract training data directly from unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-06-18T16:20:12Z)
• Synthetic Data from Diffusion Models Improve Drug Discovery Prediction [1.3686993145787065]
  Data sparsity makes data curation difficult for researchers looking to answer key research questions. We propose a novel diffusion GNN model Syngand capable of generating ligand and pharmacokinetic data end-to-end. We show initial promising results on the efficacy of the Syngand-generated synthetic target property data on downstream regression tasks with AqSolDB, LD50, and hERG central.
  arXiv  Detail & Related papers  (2024-05-06T19:09:37Z)
• Data-Efficient Molecular Generation with Hierarchical Textual Inversion [48.816943690420224]
  We introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method. HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution. Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution.
  arXiv  Detail & Related papers  (2024-05-05T08:35:23Z)
• Optimizing OOD Detection in Molecular Graphs: A Novel Approach with Diffusion Models [71.39421638547164]
  We propose to detect OOD molecules by adopting an auxiliary diffusion model-based framework, which compares similarities between input molecules and reconstructed graphs. Due to the generative bias towards reconstructing ID training samples, the similarity scores of OOD molecules will be much lower to facilitate detection. Our research pioneers an approach of Prototypical Graph Reconstruction for Molecular OOD Detection, dubbed as PGR-MOOD and hinges on three innovations.
  arXiv  Detail & Related papers  (2024-04-24T03:25:53Z)
• Objective-Agnostic Enhancement of Molecule Properties via Multi-Stage VAE [1.3597551064547502]
  Variational autoencoder (VAE) is a popular method for drug discovery and various architectures and pipelines have been proposed to improve its performance. VAE approaches are known to suffer from poor manifold recovery when the data lie on a low-dimensional manifold embedded in a higher dimensional ambient space. In this paper, we explore applying a multi-stage VAE approach, that can improve manifold recovery on a synthetic dataset, to the field of drug discovery.
  arXiv  Detail & Related papers  (2023-08-24T20:22:22Z)
• MoleHD: Ultra-Low-Cost Drug Discovery using Hyperdimensional Computing [2.7462881838152913]
  We present MoleHD, a method based on brain-inspired hyperdimensional computing (HDC) for molecular property prediction. MoleHD achieves highest ROC-AUC score on random and scaffold splits on average across 3 datasets. To the best of our knowledge, this is the first HDC-based method for drug discovery.
  arXiv  Detail & Related papers  (2021-06-05T13:33:21Z)
• Reprogramming Language Models for Molecular Representation Learning [65.00999660425731]
  We propose Representation Reprogramming via Dictionary Learning (R2DL) for adversarially reprogramming pretrained language models for molecular learning tasks. The adversarial program learns a linear transformation between a dense source model input space (language data) and a sparse target model input space (e.g., chemical and biological molecule data) using a k-SVD solver. R2DL achieves the baseline established by state of the art toxicity prediction models trained on domain-specific data and outperforms the baseline in a limited training-data setting.
  arXiv  Detail & Related papers  (2020-12-07T05:50:27Z)
• MolTrans: Molecular Interaction Transformer for Drug Target Interaction Prediction [68.5766865583049]
  Drug target interaction (DTI) prediction is a foundational task for in silico drug discovery. Recent years have witnessed promising progress for deep learning in DTI predictions. We propose a Molecular Interaction Transformer (TransMol) to address these limitations.
  arXiv  Detail & Related papers  (2020-04-23T18:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.