PhAST: Physics-Aware, Scalable, and Task-specific GNNs for Accelerated Catalyst Design

• URL: http://arxiv.org/abs/2211.12020v4
• Date: Mon, 11 Mar 2024 15:50:55 GMT
• Title: PhAST: Physics-Aware, Scalable, and Task-specific GNNs for Accelerated Catalyst Design
• Authors: Alexandre Duval, Victor Schmidt, Santiago Miret, Yoshua Bengio, Alex Hern\'andez-Garc\'ia, David Rolnick
• Abstract summary: Catalyst materials play a crucial role in the electrochemical reactions involved in industrial processes. Machine learning holds the potential to efficiently model materials properties from large amounts of data. We propose task-specific innovations applicable to most architectures, enhancing both computational efficiency and accuracy.
• Score: 102.9593507372373
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Mitigating the climate crisis requires a rapid transition towards lower-carbon energy. Catalyst materials play a crucial role in the electrochemical reactions involved in numerous industrial processes key to this transition, such as renewable energy storage and electrofuel synthesis. To reduce the energy spent on such activities, we must quickly discover more efficient catalysts to drive electrochemical reactions. Machine learning (ML) holds the potential to efficiently model materials properties from large amounts of data, accelerating electrocatalyst design. The Open Catalyst Project OC20 dataset was constructed to that end. However, ML models trained on OC20 are still neither scalable nor accurate enough for practical applications. In this paper, we propose task-specific innovations applicable to most architectures, enhancing both computational efficiency and accuracy. This includes improvements in (1) the graph creation step, (2) atom representations, (3) the energy prediction head, and (4) the force prediction head. We describe these contributions, referred to as PhAST, and evaluate them thoroughly on multiple architectures. Overall, PhAST improves energy MAE by 4 to 42$\%$ while dividing compute time by 3 to 8$\times$ depending on the targeted task/model. PhAST also enables CPU training, leading to 40$\times$ speedups in highly parallelized settings. Python package: \url{https://phast.readthedocs.io}.

Related papers

• Efficient Transformer Encoders for Mask2Former-style models [57.54752243522298]
  ECO-M2F is a strategy to self-select the number of hidden layers in the encoder conditioned on the input image. The proposed approach reduces expected encoder computational cost while maintaining performance. It is flexible in architecture configurations, and can be extended beyond the segmentation task to object detection.
  arXiv  Detail & Related papers  (2024-04-23T17:26:34Z)
• Lightweight Geometric Deep Learning for Molecular Modelling in Catalyst Discovery [0.0]
  Open Catalyst Project aims to apply advances in graph neural networks (GNNs) to accelerate progress in catalyst discovery. By implementing robust design patterns like geometric and symmetric message passing, we were able to train a GNN model that reached a MAE of 0.0748 in predicting the per-atom forces of adsorbate-surface interactions.
  arXiv  Detail & Related papers  (2024-04-05T17:13:51Z)
• Energy Transformer [64.22957136952725]
  Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory. We propose a novel architecture, called the Energy Transformer (or ET for short), that uses a sequence of attention layers that are purposely designed to minimize a specifically engineered energy function.
  arXiv  Detail & Related papers  (2023-02-14T18:51:22Z)
• Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling [51.83761266429285]
  We propose a novel Transformer architecture that takes nodes (atoms) and edges (bonds and nonbonding atom pairs) as inputs and models the interactions among them. Moleformer achieves state-of-the-art on the initial state to relaxed energy prediction of OC20 and is very competitive in QM9 on predicting quantum chemical properties.
  arXiv  Detail & Related papers  (2023-02-02T03:49:57Z)
• AdsorbML: A Leap in Efficiency for Adsorption Energy Calculations using Generalizable Machine Learning Potentials [8.636519538557001]
  We show machine learning potentials can be leveraged to identify low energy adsorbate-surface configurations more accurately and efficiently. Our algorithm provides a spectrum of trade-offs between accuracy and efficiency, with one balanced option finding the lowest energy configuration 87.36% of the time.
  arXiv  Detail & Related papers  (2022-11-29T18:54:55Z)
• Learned Force Fields Are Ready For Ground State Catalyst Discovery [60.41853574951094]
  We present evidence that learned density functional theory (DFT'') force fields are ready for ground state catalyst discovery. Key finding is that relaxation using forces from a learned potential yields structures with similar or lower energy to those relaxed using the RPBE functional in over 50% of evaluated systems. We show that a force field trained on a locally harmonic energy surface with the same minima as a target DFT energy is also able to find lower or similar energy structures in over 50% of cases.
  arXiv  Detail & Related papers  (2022-09-26T07:16:43Z)
• Carbon Emissions and Large Neural Network Training [19.233899715628073]
  We calculate the energy use and carbon footprint of several recent large models-T5, Meena, GShard, Switch Transformer, and GPT-3. We highlight the following opportunities to improve energy efficiency and CO2 equivalent emissions (CO2e) To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models.
  arXiv  Detail & Related papers  (2021-04-21T04:44:25Z)
• Efficient pre-training objectives for Transformers [84.64393460397471]
  We study several efficient pre-training objectives for Transformers-based models. We prove that eliminating the MASK token and considering the whole output during the loss are essential choices to improve performance.
  arXiv  Detail & Related papers  (2021-04-20T00:09:37Z)
• The Open Catalyst 2020 (OC20) Dataset and Community Challenges [36.556154866045894]
  Catalyst discovery and optimization is key to solving many societal and energy challenges. It remains an open challenge to build models that can generalize across both elemental compositions of surfaces and adsorbates.
  arXiv  Detail & Related papers  (2020-10-20T03:29:18Z)
• An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage [36.556154866045894]
  Conversion of renewable energy to hydrogen or methane can be scaled to nation-sized grids. To be widely adopted, this process requires cost-effective solutions to running electrochemical reactions. An open challenge is finding low-cost electrocatalysts to drive these reactions at high rates. The use of machine learning may provide a method to efficiently approximate these calculations.
  arXiv  Detail & Related papers  (2020-10-14T19:34:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.