A Materials Foundation Model via Hybrid Invariant-Equivariant Architectures

• URL: http://arxiv.org/abs/2503.05771v2
• Date: Thu, 29 May 2025 22:37:39 GMT
• Title: A Materials Foundation Model via Hybrid Invariant-Equivariant Architectures
• Authors: Keqiang Yan, Montgomery Bohde, Andrii Kryvenko, Ziyu Xiang, Kaiji Zhao, Siya Zhu, Saagar Kolachina, Doğuhan Sarıtürk, Jianwen Xie, Raymundo Arroyave, Xiaoning Qian, Xiaofeng Qian, Shuiwang Ji,
• Abstract summary: Machine learning interatomic potentials (MLIPs) can predict energy, force, and stress of materials.<n>A key design choice in MLIPs involves the trade-off between invariant and equivariant architectures.<n>HIENet is a hybrid invariant-equivariant materials interatomic potential model that integrates both invariant and equivariant message passing layers.
• Score: 53.273077346444886
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Machine learning interatomic potentials (MLIPs) can predict energy, force, and stress of materials and enable a wide range of downstream discovery tasks. A key design choice in MLIPs involves the trade-off between invariant and equivariant architectures. Invariant models offer computational efficiency but may not perform as well, especially when predicting high-order outputs. In contrast, equivariant models can capture high-order symmetries, but are computationally expensive. In this work, we propose HIENet, a hybrid invariant-equivariant materials interatomic potential model that integrates both invariant and equivariant message passing layers, while provably satisfying key physical constraints. HIENet achieves state-of-the-art performance with considerable computational speedups over prior models. Experimental results on both common benchmarks and downstream materials discovery tasks demonstrate the efficiency and effectiveness of HIENet.

Related papers

• Equivariant Graph Network Approximations of High-Degree Polynomials for Force Field Prediction [62.05532524197309]
  equivariant deep models have shown promise in accurately predicting atomic potentials and force fields in molecular dynamics simulations. In this work, we analyze the equivariant functions for equivariant architecture, and introduce a novel equivariant network, named PACE. As experimented in commonly used benchmarks, PACE demonstrates state-of-the-art performance in predicting atomic energy and force fields.
  arXiv  Detail & Related papers  (2024-11-06T19:34:40Z)
• Improving Equivariant Model Training via Constraint Relaxation [31.507956579770088]
  We propose a novel framework for improving the optimization of such models by relaxing the hard equivariance constraint during training.<n>We provide experimental results on different state-of-the-art network architectures, demonstrating how this training framework can result in equivariant models with improved generalization performance.
  arXiv  Detail & Related papers  (2024-08-23T17:35:08Z)
• Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process [8.32027826756131]
  The proposed framework is demonstrated and analyzed on three engineering case studies. It provides improved predictive accuracy over a single source model and transformed but source unaware model.
  arXiv  Detail & Related papers  (2024-07-15T22:27:04Z)
• HAMLET: Graph Transformer Neural Operator for Partial Differential Equations [13.970458554623939]
  We present a novel graph transformer framework, HAMLET, designed to address the challenges in solving partial differential equations (PDEs) using neural networks. The framework uses graph transformers with modular input encoders to directly incorporate differential equation information into the solution process. Notably, HAMLET scales effectively with increasing data complexity and noise, showcasing its robustness.
  arXiv  Detail & Related papers  (2024-02-05T21:55:24Z)
• FAENet: Frame Averaging Equivariant GNN for Materials Modeling [123.19473575281357]
  We introduce a flexible framework relying on frameaveraging (SFA) to make any model E(3)-equivariant or invariant through data transformations. We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling.
  arXiv  Detail & Related papers  (2023-04-28T21:48:31Z)
• EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning [83.11657818251447]
  We propose EqMotion, an efficient equivariant motion prediction model with invariant interaction reasoning. We conduct experiments for the proposed model on four distinct scenarios: particle dynamics, molecule dynamics, human skeleton motion prediction and pedestrian trajectory prediction. Our method achieves state-of-the-art prediction performances on all the four tasks, improving by 24.0/30.1/8.6/9.2%.
  arXiv  Detail & Related papers  (2023-03-20T05:23:46Z)
• Lorentz group equivariant autoencoders [6.858459233149096]
  Lorentz group autoencoder (LGAE) We develop an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $mathrmSO+(2,1)$, with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics.
  arXiv  Detail & Related papers  (2022-12-14T17:19:46Z)
• A Pareto-optimal compositional energy-based model for sampling and optimization of protein sequences [55.25331349436895]
  Deep generative models have emerged as a popular machine learning-based approach for inverse problems in the life sciences. These problems often require sampling new designs that satisfy multiple properties of interest in addition to learning the data distribution.
  arXiv  Detail & Related papers  (2022-10-19T19:04:45Z)
• Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach [0.0]
  Unsupervised and semi-supervised ML methods have become widely adopted across multiple areas of physics, chemistry, and materials sciences. We propose a latent Bayesian optimization (zBO) approach for the hyper parameter trajectory optimization for the unsupervised and semi-supervised ML. We demonstrate an application of this method for finding joint discrete and continuous rotationally invariant representations for MNIST and experimental data of a plasmonic nanoparticles material system.
  arXiv  Detail & Related papers  (2022-06-30T23:41:47Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• MoEfication: Conditional Computation of Transformer Models for Efficient Inference [66.56994436947441]
  Transformer-based pre-trained language models can achieve superior performance on most NLP tasks due to large parameter capacity, but also lead to huge computation cost. We explore to accelerate large-model inference by conditional computation based on the sparse activation phenomenon. We propose to transform a large model into its mixture-of-experts (MoE) version with equal model size, namely MoEfication.
  arXiv  Detail & Related papers  (2021-10-05T02:14:38Z)
• Distilling the Knowledge from Normalizing Flows [22.578033953780697]
  Normalizing flows are a powerful class of generative models demonstrating strong performance in several speech and vision problems. We propose a simple distillation approach and demonstrate its effectiveness on state-of-the-art conditional flow-based models for image super-resolution and speech synthesis.
  arXiv  Detail & Related papers  (2021-06-24T00:10:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.