Transferring a molecular foundation model for polymer property predictions

• URL: http://arxiv.org/abs/2310.16958v1
• Date: Wed, 25 Oct 2023 19:55:00 GMT
• Title: Transferring a molecular foundation model for polymer property predictions
• Authors: Pei Zhang, Logan Kearney, Debsindhu Bhowmik, Zachary Fox, Amit K. Naskar, John Gounley
• Abstract summary: Self-supervised pretraining of transformer models requires large-scale datasets. We show that using transformers pretrained on small molecules and fine-tuned on polymer properties achieve comparable accuracy to those trained on augmented polymer datasets.
• Score: 3.067983186439152
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and materials discovery. Self-supervised pretraining of transformer models requires large-scale datasets, which are often sparsely populated in topical areas such as polymer science. State-of-the-art approaches for polymers conduct data augmentation to generate additional samples but unavoidably incurs extra computational costs. In contrast, large-scale open-source datasets are available for small molecules and provide a potential solution to data scarcity through transfer learning. In this work, we show that using transformers pretrained on small molecules and fine-tuned on polymer properties achieve comparable accuracy to those trained on augmented polymer datasets for a series of benchmark prediction tasks.

Related papers

• Text-Guided Multi-Property Molecular Optimization with a Diffusion Language Model [77.50732023411811]
  We propose a text-guided multi-property molecular optimization method utilizing transformer-based diffusion language model (TransDLM) TransDLM leverages standardized chemical nomenclature as semantic representations of molecules and implicitly embeds property requirements into textual descriptions. Our approach surpasses state-of-the-art methods in optimizing molecular structural similarity and enhancing chemical properties on the benchmark dataset.
  arXiv  Detail & Related papers  (2024-10-17T14:30:27Z)
• Molecular topological deep learning for polymer property prediction [18.602659324026934]
  We develop molecular topological deep learning (Mol-TDL) for polymer property analysis. Mol-TDL incorporates both high-order interactions and multiscale properties into topological deep learning architecture.
  arXiv  Detail & Related papers  (2024-10-07T05:44:02Z)
• Transformers for molecular property prediction: Lessons learned from the past five years [0.0]
  We analyze the currently available models and explore key questions that arise when training and fine-tuning a transformer model for MPP. We address the challenges in comparing different models, emphasizing the need for standardized data splitting and robust statistical analysis.
  arXiv  Detail & Related papers  (2024-04-05T09:05:37Z)
• Molecule Design by Latent Prompt Transformer [76.2112075557233]
  This work explores the challenging problem of molecule design by framing it as a conditional generative modeling task. We propose a novel generative model comprising three components: (1) a latent vector with a learnable prior distribution; (2) a molecule generation model based on a causal Transformer, which uses the latent vector as a prompt; and (3) a property prediction model that predicts a molecule's target properties and/or constraint values using the latent prompt.
  arXiv  Detail & Related papers  (2024-02-27T03:33:23Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• TransPolymer: a Transformer-based language model for polymer property predictions [9.04563945965023]
  TransPolymer is a Transformer-based language model for polymer property prediction. Our proposed polymer tokenizer with chemical awareness enables learning representations from polymer sequences.
  arXiv  Detail & Related papers  (2022-09-03T01:29:59Z)
• Retrieval-based Controllable Molecule Generation [63.44583084888342]
  We propose a new retrieval-based framework for controllable molecule generation. We use a small set of molecules to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria. Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning.
  arXiv  Detail & Related papers  (2022-08-23T17:01:16Z)
• Geometric Transformer for End-to-End Molecule Properties Prediction [92.28929858529679]
  We introduce a Transformer-based architecture for molecule property prediction, which is able to capture the geometry of the molecule. We modify the classical positional encoder by an initial encoding of the molecule geometry, as well as a learned gated self-attention mechanism.
  arXiv  Detail & Related papers  (2021-10-26T14:14:40Z)
• Accelerating the screening of amorphous polymer electrolytes by learning to reduce random and systematic errors in molecular dynamics simulations [0.8061216572215162]
  In this work, we aim to screen amorphous polymer electrolytes which are promising candidates for the next generation lithium-ion battery technology. We demonstrate that a multi-task graph neural network can learn from a large amount of noisy, biased data and a small number of unbiased data. We screen a space of 6247 polymer electrolytes, orders of magnitude larger than previous computational studies.
  arXiv  Detail & Related papers  (2021-01-13T20:46:24Z)
• Polymer Informatics: Current Status and Critical Next Steps [1.3238373064156097]
  Surrogate models are trained on available polymer data for instant property prediction. Data-driven strategies to tackle unique challenges resulting from the extraordinary chemical and physical diversity of polymers at small and large scales are being explored. Methods to solve inverse problems, wherein polymer recommendations are made using advanced AI algorithms that meet application targets, are being investigated.
  arXiv  Detail & Related papers  (2020-11-01T14:17:22Z)
• Self-Supervised Graph Transformer on Large-Scale Molecular Data [73.3448373618865]
  We propose a novel framework, GROVER, for molecular representation learning. GROVER can learn rich structural and semantic information of molecules from enormous unlabelled molecular data. We pre-train GROVER with 100 million parameters on 10 million unlabelled molecules -- the biggest GNN and the largest training dataset in molecular representation learning.
  arXiv  Detail & Related papers  (2020-06-18T08:37:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.