An Empirical Study of Graphormer on Large-Scale Molecular Modeling Datasets

• URL: http://arxiv.org/abs/2203.06123v2
• Date: Mon, 14 Mar 2022 08:25:50 GMT
• Title: An Empirical Study of Graphormer on Large-Scale Molecular Modeling Datasets
• Authors: Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, Tie-Yan Liu
• Abstract summary: "Graphormer-V2" could attain better results on large-scale molecular modeling datasets than the vanilla one. With a global receptive field and an adaptive aggregation strategy, Graphormer is more powerful than classic message-passing-based GNNs.
• Score: 87.00711479972503
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This technical note describes the recent updates of Graphormer, including architecture design modifications, and the adaption to 3D molecular dynamics simulation. The "Graphormer-V2" could attain better results on large-scale molecular modeling datasets than the vanilla one, and the performance gain could be consistently obtained on downstream tasks. In addition, we show that with a global receptive field and an adaptive aggregation strategy, Graphormer is more powerful than classic message-passing-based GNNs. Graphormer-V2 achieves much less MAE than the vanilla Graphormer on the PCQM4M quantum chemistry dataset used in KDD Cup 2021, where the latter one won the first place in this competition. In the meanwhile, Graphormer-V2 greatly outperforms the competitors in the recent Open Catalyst Challenge, which is a competition track on NeurIPS 2021 workshop, and aims to model the catalyst-adsorbate reaction system with advanced AI models. All models could be found at \url{https://github.com/Microsoft/Graphormer}.

Related papers

• Hi-GMAE: Hierarchical Graph Masked Autoencoders [90.30572554544385]
  Hierarchical Graph Masked AutoEncoders (Hi-GMAE) Hi-GMAE is a novel multi-scale GMAE framework designed to handle the hierarchical structures within graphs. Our experiments on 15 graph datasets consistently demonstrate that Hi-GMAE outperforms 17 state-of-the-art self-supervised competitors.
  arXiv  Detail & Related papers  (2024-05-17T09:08:37Z)
• On the Scalability of GNNs for Molecular Graphs [7.402389334892391]
  Graph Neural Networks (GNNs) are yet to show the benefits of scale due to the lower efficiency of sparse operations, large data requirements, and lack of clarity about the effectiveness of various architectures. We analyze message-passing networks, graph Transformers, and hybrid architectures on the largest public collection of 2D molecular graphs. For the first time, we observe that GNNs benefit tremendously from the increasing scale of depth, width, number of molecules, number of labels, and the diversity in the pretraining datasets.
  arXiv  Detail & Related papers  (2024-04-17T17:11:31Z)
• Micro and Macro Level Graph Modeling for Graph Variational Auto-Encoders [16.302222204710276]
  This paper proposes a new multi-level framework that jointly models node-level properties and graph-level statistics. We introduce a new micro-macro training objective for graph generation that combines node-level and graph-level losses. Our experiments show that adding micro-macro modeling to the GraphVAE model improves graph quality scores up to 2 orders of magnitude on five benchmark datasets.
  arXiv  Detail & Related papers  (2022-10-30T13:45:21Z)
• MolGraph: a Python package for the implementation of molecular graphs and graph neural networks with TensorFlow and Keras [51.92255321684027]
  MolGraph is a graph neural network (GNN) package for molecular machine learning (ML) MolGraph implements a chemistry module to accommodate the generation of small molecular graphs, which can be passed to a GNN algorithm to solve a molecular ML problem. GNNs proved useful for molecular identification and improved interpretability of chromatographic retention time data.
  arXiv  Detail & Related papers  (2022-08-21T18:37:41Z)
• Graph Generative Model for Benchmarking Graph Neural Networks [73.11514658000547]
  We introduce a novel graph generative model that learns and reproduces the distribution of real-world graphs in a privacy-controlled way. Our model can successfully generate privacy-controlled, synthetic substitutes of large-scale real-world graphs that can be effectively used to benchmark GNN models.
  arXiv  Detail & Related papers  (2022-07-10T06:42:02Z)
• Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets [87.00711479972503]
  This note describes the recent updates of Graphormer. With a global receptive field and an adaptive aggregation strategy, Graphormer is more powerful than classic message-passing-based GNNs. In the meanwhile, it greatly outperforms the competitors in the recent Open Catalyst Challenge.
  arXiv  Detail & Related papers  (2022-03-09T15:40:10Z)
• Conditional Constrained Graph Variational Autoencoders for Molecule Design [70.59828655929194]
  We present Conditional Constrained Graph Variational Autoencoder (CCGVAE), a model that implements this key-idea in a state-of-the-art model. We show improved results on several evaluation metrics on two commonly adopted datasets for molecule generation.
  arXiv  Detail & Related papers  (2020-09-01T21:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.