Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials

• URL: http://arxiv.org/abs/2303.02216v2
• Date: Thu, 6 Jul 2023 03:53:17 GMT
• Title: Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• Authors: Yuyang Wang, Changwen Xu, Zijie Li, Amir Barati Farimani
• Abstract summary: We propose denoise pretraining on nonequilibrium molecular conformations to achieve more accurate and transferable GNN potential predictions. Our models pretrained on small molecules demonstrate remarkable transferability, improving performance when fine-tuned on diverse molecular systems.
• Score: 8.048439531116367
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in equivariant graph neural networks (GNNs) have made deep learning amenable to developing fast surrogate models to expensive ab initio quantum mechanics (QM) approaches for molecular potential predictions. However, building accurate and transferable potential models using GNNs remains challenging, as the data is greatly limited by the expensive computational costs and level of theory of QM methods, especially for large and complex molecular systems. In this work, we propose denoise pretraining on nonequilibrium molecular conformations to achieve more accurate and transferable GNN potential predictions. Specifically, atomic coordinates of sampled nonequilibrium conformations are perturbed by random noises and GNNs are pretrained to denoise the perturbed molecular conformations which recovers the original coordinates. Rigorous experiments on multiple benchmarks reveal that pretraining significantly improves the accuracy of neural potentials. Furthermore, we show that the proposed pretraining approach is model-agnostic, as it improves the performance of different invariant and equivariant GNNs. Notably, our models pretrained on small molecules demonstrate remarkable transferability, improving performance when fine-tuned on diverse molecular systems, including different elements, charged molecules, biomolecules, and larger systems. These results highlight the potential for leveraging denoise pretraining approaches to build more generalizable neural potentials for complex molecular systems.

Related papers

• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• Active learning of Boltzmann samplers and potential energies with quantum mechanical accuracy [1.7633275579210346]
  We develop an approach combining enhanced sampling with deep generative models and active learning of a machine learning potential. We apply this method to study the isomerization of an ultrasmall silver nanocluster, belonging to a set of systems with diverse applications in the fields of medicine and biology.
  arXiv  Detail & Related papers  (2024-01-29T19:01:31Z)
• Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning [60.02391969049972]
  We introduce a novel deep learning framework, called Distributional Graphormer (DiG), in an attempt to predict the equilibrium distribution of molecular systems. DiG employs deep neural networks to transform a simple distribution towards the equilibrium distribution, conditioned on a descriptor of a molecular system.
  arXiv  Detail & Related papers  (2023-06-08T17:12:08Z)
• MolCPT: Molecule Continuous Prompt Tuning to Generalize Molecular Representation Learning [77.31492888819935]
  We propose a novel paradigm of "pre-train, prompt, fine-tune" for molecular representation learning, named molecule continuous prompt tuning (MolCPT) MolCPT defines a motif prompting function that uses the pre-trained model to project the standalone input into an expressive prompt. Experiments on several benchmark datasets show that MolCPT efficiently generalizes pre-trained GNNs for molecular property prediction.
  arXiv  Detail & Related papers  (2022-12-20T19:32:30Z)
• Graph Neural Networks for Temperature-Dependent Activity Coefficient Prediction of Solutes in Ionic Liquids [58.720142291102135]
  We present a GNN to predict temperature-dependent infinite dilution ACs of solutes in ILs. We train the GNN on a database including more than 40,000 AC values and compare it to a state-of-the-art MCM. The GNN and MCM achieve similar high prediction performance, with the GNN additionally enabling high-quality predictions for ACs of solutions that contain ILs and solutes not considered during training.
  arXiv  Detail & Related papers  (2022-06-23T15:27:29Z)
• Accurate Machine Learned Quantum-Mechanical Force Fields for Biomolecular Simulations [51.68332623405432]
  Molecular dynamics (MD) simulations allow atomistic insights into chemical and biological processes. Recently, machine learned force fields (MLFFs) emerged as an alternative means to execute MD simulations. This work proposes a general approach to constructing accurate MLFFs for large-scale molecular simulations.
  arXiv  Detail & Related papers  (2022-05-17T13:08:28Z)
• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation [102.85440102147267]
  We propose a novel generative model named GeoDiff for molecular conformation prediction. We show that GeoDiff is superior or comparable to existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-06T09:47:01Z)
• Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments [3.1829446824051195]
  We present an improved NN architecture based on the previous GM-NN model. The improved methodology is a pre-requisite for training-heavy such as active learning or learning-on-the-fly.
  arXiv  Detail & Related papers  (2021-09-20T14:23:34Z)
• Learning neural network potentials from experimental data via Differentiable Trajectory Reweighting [0.0]
  Top-down approaches that learn neural network (NN) potentials directly from experimental data have received less attention. We present the Differentiable Trajectory Reweighting (DiffTRe) method, which bypasses differentiation through the MD simulation for time-independent observables. We show effectiveness of DiffTRe in learning NN potentials for an atomistic model of diamond and a coarse-grained model of water based on diverse experimental observables.
  arXiv  Detail & Related papers  (2021-06-02T13:10:43Z)
• SE(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials [0.17590081165362778]
  NequIP is a SE(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations. The method achieves state-of-the-art accuracy on a challenging set of diverse molecules and materials while exhibiting remarkable data efficiency.
  arXiv  Detail & Related papers  (2021-01-08T18:49:10Z)
• Symmetry-adapted graph neural networks for constructing molecular dynamics force fields [10.820190246285122]
  We develop a symmetry-adapted graph neural networks framework to construct force fields automatically for molecular dynamics simulations. We show that MDGNN accurately reproduces the results of both classical and first-principles molecular dynamics.
  arXiv  Detail & Related papers  (2021-01-08T09:32:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.