Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling

• URL: http://arxiv.org/abs/2302.00855v1
• Date: Thu, 2 Feb 2023 03:49:57 GMT
• Title: Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling
• Authors: Zheng Yuan, Yaoyun Zhang, Chuanqi Tan, Wei Wang, Fei Huang, Songfang Huang
• Abstract summary: We propose a novel Transformer architecture that takes nodes (atoms) and edges (bonds and nonbonding atom pairs) as inputs and models the interactions among them. Moleformer achieves state-of-the-art on the initial state to relaxed energy prediction of OC20 and is very competitive in QM9 on predicting quantum chemical properties.
• Score: 51.83761266429285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Molecular dynamic simulations are important in computational physics, chemistry, material, and biology. Machine learning-based methods have shown strong abilities in predicting molecular energy and properties and are much faster than DFT calculations. Molecular energy is at least related to atoms, bonds, bond angles, torsion angles, and nonbonding atom pairs. Previous Transformer models only use atoms as inputs which lack explicit modeling of the aforementioned factors. To alleviate this limitation, we propose Moleformer, a novel Transformer architecture that takes nodes (atoms) and edges (bonds and nonbonding atom pairs) as inputs and models the interactions among them using rotational and translational invariant geometry-aware spatial encoding. Proposed spatial encoding calculates relative position information including distances and angles among nodes and edges. We benchmark Moleformer on OC20 and QM9 datasets, and our model achieves state-of-the-art on the initial state to relaxed energy prediction of OC20 and is very competitive in QM9 on predicting quantum chemical properties compared to other Transformer and Graph Neural Network (GNN) methods which proves the effectiveness of the proposed geometry-aware spatial encoding in Moleformer.

Related papers

• QH9: A Quantum Hamiltonian Prediction Benchmark for QM9 Molecules [69.25826391912368]
  We generate a new Quantum Hamiltonian dataset, named as QH9, to provide precise Hamiltonian matrices for 999 or 2998 molecular dynamics trajectories. We show that current machine learning models have the capacity to predict Hamiltonian matrices for arbitrary molecules.
  arXiv  Detail & Related papers  (2023-06-15T23:39:07Z)
• ViSNet: an equivariant geometry-enhanced graph neural network with vector-scalar interactive message passing for molecules [69.05950120497221]
  We propose an equivariant geometry-enhanced graph neural network called ViSNet, which elegantly extracts geometric features and efficiently models molecular structures. Our proposed ViSNet outperforms state-of-the-art approaches on multiple MD benchmarks, including MD17, revised MD17 and MD22, and achieves excellent chemical property prediction on QM9 and Molecule3D datasets.
  arXiv  Detail & Related papers  (2022-10-29T07:12:46Z)
• Geometric Transformer for End-to-End Molecule Properties Prediction [92.28929858529679]
  We introduce a Transformer-based architecture for molecule property prediction, which is able to capture the geometry of the molecule. We modify the classical positional encoder by an initial encoding of the molecule geometry, as well as a learned gated self-attention mechanism.
  arXiv  Detail & Related papers  (2021-10-26T14:14:40Z)
• 3D-Transformer: Molecular Representation with Transformer in 3D Space [11.947499562836953]
  3D-Transformer is a variant of the Transformer for molecular representations that incorporates 3D spatial information. Our experiments show significant improvements over state-of-the-art models on the crystal property prediction task and the protein-ligand binding affinity prediction task.
  arXiv  Detail & Related papers  (2021-10-04T05:11:23Z)
• Rotation Invariant Graph Neural Networks using Spin Convolutions [28.4962005849904]
  Machine learning approaches have the potential to approximate Density Functional Theory (DFT) in a computationally efficient manner. We introduce a novel approach to modeling angular information between sets of neighboring atoms in a graph neural network. Results are demonstrated on the large-scale Open Catalyst 2020 dataset.
  arXiv  Detail & Related papers  (2021-06-17T14:59:34Z)
• GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles [60.12186997181117]
  Prediction of a molecule's 3D conformer ensemble from the molecular graph holds a key role in areas of cheminformatics and drug discovery. Existing generative models have several drawbacks including lack of modeling important molecular geometry elements. We propose GeoMol, an end-to-end, non-autoregressive and SE(3)-invariant machine learning approach to generate 3D conformers.
  arXiv  Detail & Related papers  (2021-06-08T14:17:59Z)
• End-to-End Differentiable Molecular Mechanics Force Field Construction [0.5269923665485903]
  We propose an alternative approach that uses graph neural networks to perceive chemical environments. The entire process is modular and end-to-end differentiable with respect to model parameters. We show that this approach is not only sufficiently to reproduce legacy atom types, but that it can learn to accurately reproduce and extend existing molecular mechanics force fields.
  arXiv  Detail & Related papers  (2020-10-02T20:59:46Z)
• Graph Neural Network for Hamiltonian-Based Material Property Prediction [56.94118357003096]
  We present and compare several different graph convolution networks that are able to predict the band gap for inorganic materials. The models are developed to incorporate two different features: the information of each orbital itself and the interaction between each other. The results show that our model can get a promising prediction accuracy with cross-validation.
  arXiv  Detail & Related papers  (2020-05-27T13:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.