A Materials Foundation Model via Hybrid Invariant-Equivariant Architectures

• URL: http://arxiv.org/abs/2503.05771v1
• Date: Tue, 25 Feb 2025 18:01:05 GMT
• Title: A Materials Foundation Model via Hybrid Invariant-Equivariant Architectures
• Authors: Keqiang Yan, Montgomery Bohde, Andrii Kryvenko, Ziyu Xiang, Kaiji Zhao, Siya Zhu, Saagar Kolachina, Doğuhan Sarıtürk, Jianwen Xie, Raymundo Arroyave, Xiaoning Qian, Xiaofeng Qian, Shuiwang Ji,
• Abstract summary: HIENet is a hybrid invariant-equivariant foundation model that integrates both invariant and equivariant message passing layers.<n>Results on both common benchmarks and downstream materials discovery tasks demonstrate the efficiency and effectiveness of HIENet.
• Score: 53.273077346444886
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Materials foundation models can predict energy, force, and stress of materials and enable a wide range of downstream discovery tasks. A key design choice involves the trade-off between invariant and equivariant architectures. Invariant models offer computational efficiency but may not perform well when predicting high-order outputs. In contrast, equivariant models can capture high-order symmetries, but are computationally expensive. In this work, we propose HIENet, a hybrid invariant-equivariant foundation model that integrates both invariant and equivariant message passing layers. HIENet is designed to achieve superior performance with considerable computational speedups over prior models. Experimental results on both common benchmarks and downstream materials discovery tasks demonstrate the efficiency and effectiveness of HIENet.

Related papers

• Improving Equivariant Model Training via Constraint Relaxation [31.507956579770088]
  We propose a novel framework for improving the optimization of such models by relaxing the hard equivariance constraint during training.<n>We provide experimental results on different state-of-the-art network architectures, demonstrating how this training framework can result in equivariant models with improved generalization performance.
  arXiv  Detail & Related papers  (2024-08-23T17:35:08Z)
• Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process [8.32027826756131]
  The proposed framework is demonstrated and analyzed on three engineering case studies. It provides improved predictive accuracy over a single source model and transformed but source unaware model.
  arXiv  Detail & Related papers  (2024-07-15T22:27:04Z)
• FAENet: Frame Averaging Equivariant GNN for Materials Modeling [123.19473575281357]
  We introduce a flexible framework relying on frameaveraging (SFA) to make any model E(3)-equivariant or invariant through data transformations. We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling.
  arXiv  Detail & Related papers  (2023-04-28T21:48:31Z)
• EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning [83.11657818251447]
  We propose EqMotion, an efficient equivariant motion prediction model with invariant interaction reasoning. We conduct experiments for the proposed model on four distinct scenarios: particle dynamics, molecule dynamics, human skeleton motion prediction and pedestrian trajectory prediction. Our method achieves state-of-the-art prediction performances on all the four tasks, improving by 24.0/30.1/8.6/9.2%.
  arXiv  Detail & Related papers  (2023-03-20T05:23:46Z)
• A Pareto-optimal compositional energy-based model for sampling and optimization of protein sequences [55.25331349436895]
  Deep generative models have emerged as a popular machine learning-based approach for inverse problems in the life sciences. These problems often require sampling new designs that satisfy multiple properties of interest in addition to learning the data distribution.
  arXiv  Detail & Related papers  (2022-10-19T19:04:45Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• MoEfication: Conditional Computation of Transformer Models for Efficient Inference [66.56994436947441]
  Transformer-based pre-trained language models can achieve superior performance on most NLP tasks due to large parameter capacity, but also lead to huge computation cost. We explore to accelerate large-model inference by conditional computation based on the sparse activation phenomenon. We propose to transform a large model into its mixture-of-experts (MoE) version with equal model size, namely MoEfication.
  arXiv  Detail & Related papers  (2021-10-05T02:14:38Z)
• Distilling the Knowledge from Normalizing Flows [22.578033953780697]
  Normalizing flows are a powerful class of generative models demonstrating strong performance in several speech and vision problems. We propose a simple distillation approach and demonstrate its effectiveness on state-of-the-art conditional flow-based models for image super-resolution and speech synthesis.
  arXiv  Detail & Related papers  (2021-06-24T00:10:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.