Synergistic Fusion of Graph and Transformer Features for Enhanced Molecular Property Prediction

• URL: http://arxiv.org/abs/2310.03027v1
• Date: Fri, 25 Aug 2023 14:47:46 GMT
• Title: Synergistic Fusion of Graph and Transformer Features for Enhanced Molecular Property Prediction
• Authors: M V Sai Prakash, Siddartha Reddy N, Ganesh Parab, Varun V, Vishal Vaddina, Saisubramaniam Gopalakrishnan
• Abstract summary: We propose a novel approach that combines pre-trained features from GNNs and Transformers. This approach provides a comprehensive molecular representation, capturing both the global molecule structure and the individual atom characteristics. Experimental results on MoleculeNet benchmarks demonstrate superior performance, surpassing previous models in 5 out of 7 classification datasets and 4 out of 6 regression datasets.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Molecular property prediction is a critical task in computational drug discovery. While recent advances in Graph Neural Networks (GNNs) and Transformers have shown to be effective and promising, they face the following limitations: Transformer self-attention does not explicitly consider the underlying molecule structure while GNN feature representation alone is not sufficient to capture granular and hidden interactions and characteristics that distinguish similar molecules. To address these limitations, we propose SYN- FUSION, a novel approach that synergistically combines pre-trained features from GNNs and Transformers. This approach provides a comprehensive molecular representation, capturing both the global molecule structure and the individual atom characteristics. Experimental results on MoleculeNet benchmarks demonstrate superior performance, surpassing previous models in 5 out of 7 classification datasets and 4 out of 6 regression datasets. The performance of SYN-FUSION has been compared with other Graph-Transformer models that have been jointly trained using a combination of transformer and graph features, and it is found that our approach is on par with those models in terms of performance. Extensive analysis of the learned fusion model across aspects such as loss, latent space, and weight distribution further validates the effectiveness of SYN-FUSION. Finally, an ablation study unequivocally demonstrates that the synergy achieved by SYN-FUSION surpasses the performance of its individual model components and their ensemble, offering a substantial improvement in predicting molecular properties.

Related papers

• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• YZS-model: A Predictive Model for Organic Drug Solubility Based on Graph Convolutional Networks and Transformer-Attention [9.018408514318631]
  Traditional methods often miss complex molecular structures, leading to inaccuracies. We introduce the YZS-Model, a deep learning framework integrating Graph Convolutional Networks (GCN), Transformer architectures, and Long Short-Term Memory (LSTM) networks. YZS-Model achieved an $R2$ of 0.59 and an RMSE of 0.57, outperforming benchmark models.
  arXiv  Detail & Related papers  (2024-06-27T12:40:29Z)
• UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment [51.49238426241974]
  This paper introduces UAlign, a template-free graph-to-sequence pipeline for retrosynthesis prediction. By combining graph neural networks and Transformers, our method can more effectively leverage the inherent graph structure of molecules.
  arXiv  Detail & Related papers  (2024-03-25T03:23:03Z)
• Molecule Design by Latent Prompt Transformer [76.2112075557233]
  This work explores the challenging problem of molecule design by framing it as a conditional generative modeling task. We propose a novel generative model comprising three components: (1) a latent vector with a learnable prior distribution; (2) a molecule generation model based on a causal Transformer, which uses the latent vector as a prompt; and (3) a property prediction model that predicts a molecule's target properties and/or constraint values using the latent prompt.
  arXiv  Detail & Related papers  (2024-02-27T03:33:23Z)
• MultiModal-Learning for Predicting Molecular Properties: A Framework Based on Image and Graph Structures [2.5563339057415218]
  MolIG is a novel MultiModaL molecular pre-training framework for predicting molecular properties based on Image and Graph structures. It amalgamates the strengths of both molecular representation forms. It exhibits enhanced performance in downstream tasks pertaining to molecular property prediction within benchmark groups.
  arXiv  Detail & Related papers  (2023-11-28T10:28:35Z)
• Atomic and Subgraph-aware Bilateral Aggregation for Molecular Representation Learning [57.670845619155195]
  We introduce a new model for molecular representation learning called the Atomic and Subgraph-aware Bilateral Aggregation (ASBA) ASBA addresses the limitations of previous atom-wise and subgraph-wise models by incorporating both types of information. Our method offers a more comprehensive way to learn representations for molecular property prediction and has broad potential in drug and material discovery applications.
  arXiv  Detail & Related papers  (2023-05-22T00:56:00Z)
• Dynamic Molecular Graph-based Implementation for Biophysical Properties Prediction [9.112532782451233]
  We propose a novel approach based on the transformer model utilizing GNNs for characterizing dynamic features of protein-ligand interactions. Our message passing transformer pre-trains on a set of molecular dynamic data based off of physics-based simulations to learn coordinate construction and make binding probability and affinity predictions.
  arXiv  Detail & Related papers  (2022-12-20T04:21:19Z)
• Graph neural networks for the prediction of molecular structure-property relationships [59.11160990637615]
  Graph neural networks (GNNs) are a novel machine learning method that directly work on the molecular graph. GNNs allow to learn properties in an end-to-end fashion, thereby avoiding the need for informative descriptors. We describe the fundamentals of GNNs and demonstrate the application of GNNs via two examples for molecular property prediction.
  arXiv  Detail & Related papers  (2022-07-25T11:30:44Z)
• Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction [2.5655440962401617]
  We describe a novel Graph2SMILES model that combines the power of Transformer models for text generation with the permutation invariance of molecular graph encoders. As an end-to-end architecture, Graph2SMILES can be used as a drop-in replacement for the Transformer in any task involving molecule(s)-to-molecule(s) transformations.
  arXiv  Detail & Related papers  (2021-10-19T01:23:15Z)
• Few-Shot Graph Learning for Molecular Property Prediction [46.60746023179724]
  We propose Meta-MGNN, a novel model for few-shot molecular property prediction. To exploit unlabeled molecular information, Meta-MGNN further incorporates molecular structure, attribute based self-supervised modules and self-attentive task weights. Extensive experiments on two public multi-property datasets demonstrate that Meta-MGNN outperforms a variety of state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-16T01:55:34Z)
• Multi-View Graph Neural Networks for Molecular Property Prediction [67.54644592806876]
  We present Multi-View Graph Neural Network (MV-GNN), a multi-view message passing architecture. In MV-GNN, we introduce a shared self-attentive readout component and disagreement loss to stabilize the training process. We further boost the expressive power of MV-GNN by proposing a cross-dependent message passing scheme.
  arXiv  Detail & Related papers  (2020-05-17T04:46:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.