Lightweight Geometric Deep Learning for Molecular Modelling in Catalyst Discovery

• URL: http://arxiv.org/abs/2404.10003v1
• Date: Fri, 5 Apr 2024 17:13:51 GMT
• Title: Lightweight Geometric Deep Learning for Molecular Modelling in Catalyst Discovery
• Authors: Patrick Geitner,
• Abstract summary: Open Catalyst Project aims to apply advances in graph neural networks (GNNs) to accelerate progress in catalyst discovery. By implementing robust design patterns like geometric and symmetric message passing, we were able to train a GNN model that reached a MAE of 0.0748 in predicting the per-atom forces of adsorbate-surface interactions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: New technology for energy storage is necessary for the large-scale adoption of renewable energy sources like wind and solar. The ability to discover suitable catalysts is crucial for making energy storage more cost-effective and scalable. The Open Catalyst Project aims to apply advances in graph neural networks (GNNs) to accelerate progress in catalyst discovery, replacing Density Functional Theory-based (DFT) approaches that are computationally burdensome. Current approaches involve scaling GNNs to over 1 billion parameters, pushing the problem out of reach for a vast majority of machine learning practitioner around the world. This study aims to evaluate the performance and insights gained from using more lightweight approaches for this task that are more approachable for smaller teams to encourage participation from individuals from diverse backgrounds. By implementing robust design patterns like geometric and symmetric message passing, we were able to train a GNN model that reached a MAE of 0.0748 in predicting the per-atom forces of adsorbate-surface interactions, rivaling established model architectures like SchNet and DimeNet++ while using only a fraction of trainable parameters.

Related papers

• ssProp: Energy-Efficient Training for Convolutional Neural Networks with Scheduled Sparse Back Propagation [4.77407121905745]
  Back-propagation (BP) is a major source of computational expense during training deep learning models. We propose a general, energy-efficient convolution module that can be seamlessly integrated into any deep learning architecture.
  arXiv  Detail & Related papers  (2024-08-22T17:22:59Z)
• Adaptive Catalyst Discovery Using Multicriteria Bayesian Optimization with Representation Learning [17.00084254889438]
  High-performance catalysts are crucial for sustainable energy conversion and human health. The discovery of catalysts faces challenges due to the absence of efficient approaches to navigating vast and high-dimensional structure and composition spaces.
  arXiv  Detail & Related papers  (2024-04-18T18:11:06Z)
• Towards Meta-Pruning via Optimal Transport [64.6060250923073]
  This paper introduces a novel approach named Intra-Fusion, challenging the prevailing pruning paradigm. We leverage the concepts of model fusion and Optimal Transport to arrive at a more effective sparse model representation. We benchmark our results for various networks on commonly used datasets such as CIFAR-10, CIFAR-100, and ImageNet.
  arXiv  Detail & Related papers  (2024-02-12T17:50:56Z)
• Gradual Optimization Learning for Conformational Energy Minimization [69.36925478047682]
  Gradual Optimization Learning Framework (GOLF) for energy minimization with neural networks significantly reduces the required additional data. Our results demonstrate that the neural network trained with GOLF performs on par with the oracle on a benchmark of diverse drug-like molecules.
  arXiv  Detail & Related papers  (2023-11-05T11:48:08Z)
• SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [51.6943465041708]
  Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency. We contribute a full-stack toolkit for pre-processing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips.
  arXiv  Detail & Related papers  (2023-10-25T13:15:17Z)
• Energy Transformer [64.22957136952725]
  Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory. We propose a novel architecture, called the Energy Transformer (or ET for short), that uses a sequence of attention layers that are purposely designed to minimize a specifically engineered energy function.
  arXiv  Detail & Related papers  (2023-02-14T18:51:22Z)
• Boosting Heterogeneous Catalyst Discovery by Structurally Constrained Deep Learning Models [0.0]
  Deep learning approaches such as graph neural networks (GNNs) open new opportunity to significantly extend scope for modelling novel high-performance catalysts. Here we present embedding improvement for GNN that has been modified by Voronoi tesselation. We show that a sensible choice of data can decrease the error to values above physically-based 20 meV per atom threshold.
  arXiv  Detail & Related papers  (2022-07-11T17:01:28Z)
• Powerpropagation: A sparsity inducing weight reparameterisation [65.85142037667065]
  We introduce Powerpropagation, a new weight- parameterisation for neural networks that leads to inherently sparse models. Models trained in this manner exhibit similar performance, but have a distribution with markedly higher density at zero, allowing more parameters to be pruned safely. Here, we combine Powerpropagation with a traditional weight-pruning technique as well as recent state-of-the-art sparse-to-sparse algorithms, showing superior performance on the ImageNet benchmark.
  arXiv  Detail & Related papers  (2021-10-01T10:03:57Z)
• Rotation Invariant Graph Neural Networks using Spin Convolutions [28.4962005849904]
  Machine learning approaches have the potential to approximate Density Functional Theory (DFT) in a computationally efficient manner. We introduce a novel approach to modeling angular information between sets of neighboring atoms in a graph neural network. Results are demonstrated on the large-scale Open Catalyst 2020 dataset.
  arXiv  Detail & Related papers  (2021-06-17T14:59:34Z)
• Energy-Efficient and Federated Meta-Learning via Projected Stochastic Gradient Ascent [79.58680275615752]
  We propose an energy-efficient federated meta-learning framework. We assume each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model.
  arXiv  Detail & Related papers  (2021-05-31T08:15:44Z)
• The Open Catalyst 2020 (OC20) Dataset and Community Challenges [36.556154866045894]
  Catalyst discovery and optimization is key to solving many societal and energy challenges. It remains an open challenge to build models that can generalize across both elemental compositions of surfaces and adsorbates.
  arXiv  Detail & Related papers  (2020-10-20T03:29:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.