MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction

• URL: http://arxiv.org/abs/2406.12950v2
• Date: Fri, 18 Oct 2024 12:19:41 GMT
• Title: MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction
• Authors: Yuyan Liu, Sirui Ding, Sheng Zhou, Wenqi Fan, Qiaoyu Tan,
• Abstract summary: We present MolecularGPT for few-shot molecular property prediction. It exhibits competitive in-context reasoning capabilities across 10 downstream evaluation datasets. It also excels state-of-the-art LLM baselines by up to 15.7% increase on classification accuracy and decrease of 17.9 on regression metrics (e.g., RMSE) under zero-shot.
• Score: 20.92779223471019
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Molecular property prediction (MPP) is a fundamental and crucial task in drug discovery. However, prior methods are limited by the requirement for a large number of labeled molecules and their restricted ability to generalize for unseen and new tasks, both of which are essential for real-world applications. To address these challenges, we present MolecularGPT for few-shot MPP. From a perspective on instruction tuning, we fine-tune large language models (LLMs) based on curated molecular instructions spanning over 1000 property prediction tasks. This enables building a versatile and specialized LLM that can be adapted to novel MPP tasks without any fine-tuning through zero- and few-shot in-context learning (ICL). MolecularGPT exhibits competitive in-context reasoning capabilities across 10 downstream evaluation datasets, setting new benchmarks for few-shot molecular prediction tasks. More importantly, with just two-shot examples, MolecularGPT can outperform standard supervised graph neural network methods on 4 out of 7 datasets. It also excels state-of-the-art LLM baselines by up to 15.7% increase on classification accuracy and decrease of 17.9 on regression metrics (e.g., RMSE) under zero-shot. This study demonstrates the potential of LLMs as effective few-shot molecular property predictors. The code is available at https://github.com/NYUSHCS/MolecularGPT.

Related papers

• MoRA: On-the-fly Molecule-aware Low-Rank Adaptation Framework for LLM-based Multi-Modal Molecular Assistant [23.60380089071549]
  We propose an instance-specific parameter space alignment approach for each molecule on-the-fly.<n>MoRA produces a unique set of low-rank adaptation weights for each input molecular graph.<n>Experiments demonstrate that MoRA's instance-specific dynamic adaptation outperforms statically adapted baselines.
  arXiv  Detail & Related papers  (2025-10-14T07:54:43Z)
• Reasoning-Enhanced Large Language Models for Molecular Property Prediction [19.593493317167646]
  Molecular property prediction is crucial for drug discovery and materials science.<n>Existing approaches suffer from limited interpretability, poor cross-task generalization, and lack of chemical reasoning capabilities.<n>We propose MPPReasoner, a multimodal large language model that incorporates chemical reasoning for molecular property prediction.
  arXiv  Detail & Related papers  (2025-10-11T15:05:45Z)
• Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization [8.846705148987652]
  We introduce Mol-LLM, the first multimodal generalist model that handles a broad spectrum of molecular tasks.<n> Mol-LLM attains state-of-the-art or comparable results across the most comprehensive molecular-LLM benchmark.
  arXiv  Detail & Related papers  (2025-02-05T01:14:12Z)
• Two-Stage Pretraining for Molecular Property Prediction in the Wild [38.31911435361748]
  We introduce MoleVers, a versatile pretrained model designed for various types of molecular property prediction in the wild. MoleVers learns representations from large unlabeled datasets via masked atom prediction and dynamic denoising. In the second stage, MoleVers is further pretrained using auxiliary labels obtained with inexpensive computational methods.
  arXiv  Detail & Related papers  (2024-11-05T22:36:17Z)
• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• Can Large Language Models Empower Molecular Property Prediction? [16.5246941211725]
  Molecular property prediction has gained significant attention due to its transformative potential in scientific disciplines. Recently, the rapid development of Large Language Models (LLMs) has revolutionized the field of NLP. In this work, we advance towards this objective through two perspectives: zero/few-shot molecular classification, and using the new explanations generated by LLMs as representations of molecules.
  arXiv  Detail & Related papers  (2023-07-14T16:06:42Z)
• Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective [53.300288393173204]
  Large Language Models (LLMs) have shown remarkable performance in various cross-modal tasks. In this work, we propose an In-context Few-Shot Molecule Learning paradigm for molecule-caption translation. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation.
  arXiv  Detail & Related papers  (2023-06-11T08:16:25Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• MolCPT: Molecule Continuous Prompt Tuning to Generalize Molecular Representation Learning [77.31492888819935]
  We propose a novel paradigm of "pre-train, prompt, fine-tune" for molecular representation learning, named molecule continuous prompt tuning (MolCPT) MolCPT defines a motif prompting function that uses the pre-trained model to project the standalone input into an expressive prompt. Experiments on several benchmark datasets show that MolCPT efficiently generalizes pre-trained GNNs for molecular property prediction.
  arXiv  Detail & Related papers  (2022-12-20T19:32:30Z)
• Do Large Scale Molecular Language Representations Capture Important Structural Information? [31.76876206167457]
  We present molecular embeddings obtained by training an efficient transformer encoder model, referred to as MoLFormer. Experiments show that the learned molecular representation performs competitively, when compared to graph-based and fingerprint-based supervised learning baselines.
  arXiv  Detail & Related papers  (2021-06-17T14:33:55Z)
• Few-Shot Graph Learning for Molecular Property Prediction [46.60746023179724]
  We propose Meta-MGNN, a novel model for few-shot molecular property prediction. To exploit unlabeled molecular information, Meta-MGNN further incorporates molecular structure, attribute based self-supervised modules and self-attentive task weights. Extensive experiments on two public multi-property datasets demonstrate that Meta-MGNN outperforms a variety of state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-16T01:55:34Z)
• Learn molecular representations from large-scale unlabeled molecules for drug discovery [19.222413268610808]
  Molecular Pre-training Graph-based deep learning framework, named MPG, leans molecular representations from large-scale unlabeled molecules. MolGNet can capture valuable chemistry insights to produce interpretable representation. MPG is promising to become a novel approach in the drug discovery pipeline.
  arXiv  Detail & Related papers  (2020-12-21T08:21:49Z)
• MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization [51.00815310242277]
  generative models and reinforcement learning approaches made initial success, but still face difficulties in simultaneously optimizing multiple drug properties. We propose the MultI-constraint MOlecule SAmpling (MIMOSA) approach, a sampling framework to use input molecule as an initial guess and sample molecules from the target distribution.
  arXiv  Detail & Related papers  (2020-10-05T20:18:42Z)
• Self-Supervised Graph Transformer on Large-Scale Molecular Data [73.3448373618865]
  We propose a novel framework, GROVER, for molecular representation learning. GROVER can learn rich structural and semantic information of molecules from enormous unlabelled molecular data. We pre-train GROVER with 100 million parameters on 10 million unlabelled molecules -- the biggest GNN and the largest training dataset in molecular representation learning.
  arXiv  Detail & Related papers  (2020-06-18T08:37:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.