In-Context Learning for Few-Shot Molecular Property Prediction
- URL: http://arxiv.org/abs/2310.08863v1
- Date: Fri, 13 Oct 2023 05:12:48 GMT
- Title: In-Context Learning for Few-Shot Molecular Property Prediction
- Authors: Christopher Fifty, Jure Leskovec, Sebastian Thrun
- Abstract summary: In this paper, we adapt the concepts underpinning in-context learning to develop a new algorithm for few-shot molecular property prediction.
Our approach learns to predict molecular properties from a context of (molecule, property measurement) pairs and rapidly adapts to new properties without fine-tuning.
- Score: 56.67309268480843
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In-context learning has become an important approach for few-shot learning in
Large Language Models because of its ability to rapidly adapt to new tasks
without fine-tuning model parameters. However, it is restricted to applications
in natural language and inapplicable to other domains. In this paper, we adapt
the concepts underpinning in-context learning to develop a new algorithm for
few-shot molecular property prediction. Our approach learns to predict
molecular properties from a context of (molecule, property measurement) pairs
and rapidly adapts to new properties without fine-tuning. On the FS-Mol and
BACE molecular property prediction benchmarks, we find this method surpasses
the performance of recent meta-learning algorithms at small support sizes and
is competitive with the best methods at large support sizes.
Related papers
- Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning [0.0]
We introduce a Multi-Modal Fusion (MMF) framework that harnesses the analytical prowess of Graph Neural Networks (GNNs) and the linguistic generative and predictive abilities of Large Language Models (LLMs)
Our framework combines the effectiveness of GNNs in modeling graph-structured data with the zero-shot and few-shot learning capabilities of LLMs, enabling improved predictions while reducing the risk of overfitting.
arXiv Detail & Related papers (2024-08-27T11:10:39Z) - Molecular Graph Representation Learning Integrating Large Language Models with Domain-specific Small Models [12.744381867301353]
We propose a novel Molecular Graph representation learning framework that integrates Large language models and Domain-specific small models.
We employ a multi-modal alignment method to coordinate various modalities, including molecular graphs and their corresponding descriptive texts, to guide the pre-training of molecular representations.
arXiv Detail & Related papers (2024-08-19T16:11:59Z) - Ensemble Model With Bert,Roberta and Xlnet For Molecular property prediction [0.0]
This paper presents a novel approach for predicting molecular properties with high accuracy without the need for extensive pre-training.
We employ ensemble learning and supervised fine-tuning of BERT, RoBERTa, and XLNet.
This innovation provides a cost-effective and resource-efficient solution, potentially advancing further research in the molecular domain.
arXiv Detail & Related papers (2024-05-30T10:03:58Z) - SINC: Self-Supervised In-Context Learning for Vision-Language Tasks [64.44336003123102]
We propose a framework to enable in-context learning in large language models.
A meta-model can learn on self-supervised prompts consisting of tailored demonstrations.
Experiments show that SINC outperforms gradient-based methods in various vision-language tasks.
arXiv Detail & Related papers (2023-07-15T08:33:08Z) - Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular
Property Prediction [53.06671763877109]
We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction.
Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations.
On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
arXiv Detail & Related papers (2023-02-04T01:32:40Z) - Few-Shot Graph Learning for Molecular Property Prediction [46.60746023179724]
We propose Meta-MGNN, a novel model for few-shot molecular property prediction.
To exploit unlabeled molecular information, Meta-MGNN further incorporates molecular structure, attribute based self-supervised modules and self-attentive task weights.
Extensive experiments on two public multi-property datasets demonstrate that Meta-MGNN outperforms a variety of state-of-the-art methods.
arXiv Detail & Related papers (2021-02-16T01:55:34Z) - Reprogramming Language Models for Molecular Representation Learning [65.00999660425731]
We propose Representation Reprogramming via Dictionary Learning (R2DL) for adversarially reprogramming pretrained language models for molecular learning tasks.
The adversarial program learns a linear transformation between a dense source model input space (language data) and a sparse target model input space (e.g., chemical and biological molecule data) using a k-SVD solver.
R2DL achieves the baseline established by state of the art toxicity prediction models trained on domain-specific data and outperforms the baseline in a limited training-data setting.
arXiv Detail & Related papers (2020-12-07T05:50:27Z) - Reinforced Molecular Optimization with Neighborhood-Controlled Grammars [63.84003497770347]
We propose MNCE-RL, a graph convolutional policy network for molecular optimization.
We extend the original neighborhood-controlled embedding grammars to make them applicable to molecular graph generation.
We show that our approach achieves state-of-the-art performance in a diverse range of molecular optimization tasks.
arXiv Detail & Related papers (2020-11-14T05:42:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.