GNN-SKAN: Harnessing the Power of SwallowKAN to Advance Molecular Representation Learning with GNNs

• URL: http://arxiv.org/abs/2408.01018v4
• Date: Sat, 09 Nov 2024 14:22:03 GMT
• Title: GNN-SKAN: Harnessing the Power of SwallowKAN to Advance Molecular Representation Learning with GNNs
• Authors: Ruifeng Li, Mingqian Li, Wei Liu, Hongyang Chen,
• Abstract summary: We introduce a new class of GNNs that integrates the Kolmogorov-Arnold Networks (KANs) KANs are known for their robust data-fitting capabilities and high accuracy in small-scale AI + Science tasks. We propose a new class of GNNs, GNN-SKAN, and its augmented variant, GNN-SKAN+, which incorporates a SKAN-based classifier to further boost performance.
• Score: 19.019980841275366
• License:
• Abstract: Effective molecular representation learning is crucial for advancing molecular property prediction and drug design. Mainstream molecular representation learning approaches are based on Graph Neural Networks (GNNs). However, these approaches struggle with three significant challenges: insufficient annotations, molecular diversity, and architectural limitations such as over-squashing, which leads to the loss of critical structural details. To address these challenges, we introduce a new class of GNNs that integrates the Kolmogorov-Arnold Networks (KANs), known for their robust data-fitting capabilities and high accuracy in small-scale AI + Science tasks. By incorporating KANs into GNNs, our model enhances the representation of molecular structures. We further advance this approach with a variant called SwallowKAN (SKAN), which employs adaptive Radial Basis Functions (RBFs) as the core of the non-linear neurons. This innovation improves both computational efficiency and adaptability to diverse molecular structures. Building on the strengths of SKAN, we propose a new class of GNNs, GNN-SKAN, and its augmented variant, GNN-SKAN+, which incorporates a SKAN-based classifier to further boost performance. To our knowledge, this is the first work to integrate KANs into GNN architectures tailored for molecular representation learning. Experiments across 6 classification datasets, 6 regression datasets, and 4 few-shot learning datasets demonstrate that our approach achieves new state-of-the-art performance in terms of accuracy and computational cost.

Related papers

• Neural P$^3$M: A Long-Range Interaction Modeling Enhancer for Geometric GNNs [66.98487644676906]
  We introduce Neural P$3$M, a versatile enhancer of geometric GNNs to expand the scope of their capabilities. It exhibits flexibility across a wide range of molecular systems and demonstrates remarkable accuracy in predicting energies and forces. It also achieves an average improvement of 22% on the OE62 dataset while integrating with various architectures.
  arXiv  Detail & Related papers  (2024-09-26T08:16:59Z)
• Accelerating Molecular Graph Neural Networks via Knowledge Distillation [1.9116784879310031]
  Recent advances in graph neural networks (GNNs) have enabled more comprehensive modeling of molecules and molecular systems. As the field has been progressing to bigger and more complex architectures, state-of-the-art GNNs have become largely prohibitive for many large-scale applications. We devise KD strategies that facilitate the distillation of hidden representations in directional and equivariant GNNs, and evaluate their performance on the regression task of energy and force prediction.
  arXiv  Detail & Related papers  (2023-06-26T16:24:31Z)
• HiGNN: Hierarchical Informative Graph Neural Networks for Molecular Property Prediction Equipped with Feature-Wise Attention [5.735627221409312]
  We propose a well-designed hierarchical informative graph neural networks framework (termed HiGNN) for predicting molecular property. Experiments demonstrate that HiGNN achieves state-of-the-art predictive performance on many challenging drug discovery-associated benchmark datasets.
  arXiv  Detail & Related papers  (2022-08-30T05:16:15Z)
• Graph neural networks for the prediction of molecular structure-property relationships [59.11160990637615]
  Graph neural networks (GNNs) are a novel machine learning method that directly work on the molecular graph. GNNs allow to learn properties in an end-to-end fashion, thereby avoiding the need for informative descriptors. We describe the fundamentals of GNNs and demonstrate the application of GNNs via two examples for molecular property prediction.
  arXiv  Detail & Related papers  (2022-07-25T11:30:44Z)
• Image-Like Graph Representations for Improved Molecular Property Prediction [7.119677737397071]
  We propose a new intrinsic molecular representation that bypasses the need for GNNs entirely, dubbed CubeMol. Our fixed-dimensional representation, when paired with a transformer model, exceeds the performance of state-of-the-art GNN models and provides a path for scalability.
  arXiv  Detail & Related papers  (2021-11-20T22:39:11Z)
• Evolutionary Architecture Search for Graph Neural Networks [23.691915813153496]
  We propose a novel AutoML framework through the evolution of individual models in a large Graph Neural Networks (GNN) architecture space. To the best of our knowledge, this is the first work to introduce and evaluate evolutionary architecture search for GNN models.
  arXiv  Detail & Related papers  (2020-09-21T22:11:53Z)
• Neural Networks Enhancement with Logical Knowledge [83.9217787335878]
  We propose an extension of KENN for relational data. The results show that KENN is capable of increasing the performances of the underlying neural network even in the presence relational data.
  arXiv  Detail & Related papers  (2020-09-13T21:12:20Z)
• Graph Neural Network Architecture Search for Molecular Property Prediction [1.0965065178451106]
  We develop an NAS approach to automate the design and development of graph neural networks (GNNs) for molecular property prediction. Specifically, we focus on automated development of message-passing neural networks (MPNNs) to predict the molecular properties of small molecules in quantum mechanics and physical chemistry data sets.
  arXiv  Detail & Related papers  (2020-08-27T15:30:57Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Self-Supervised Graph Transformer on Large-Scale Molecular Data [73.3448373618865]
  We propose a novel framework, GROVER, for molecular representation learning. GROVER can learn rich structural and semantic information of molecules from enormous unlabelled molecular data. We pre-train GROVER with 100 million parameters on 10 million unlabelled molecules -- the biggest GNN and the largest training dataset in molecular representation learning.
  arXiv  Detail & Related papers  (2020-06-18T08:37:04Z)
• Multi-View Graph Neural Networks for Molecular Property Prediction [67.54644592806876]
  We present Multi-View Graph Neural Network (MV-GNN), a multi-view message passing architecture. In MV-GNN, we introduce a shared self-attentive readout component and disagreement loss to stabilize the training process. We further boost the expressive power of MV-GNN by proposing a cross-dependent message passing scheme.
  arXiv  Detail & Related papers  (2020-05-17T04:46:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.