Leveraging neural network interatomic potentials for a foundation model of chemistry

• URL: http://arxiv.org/abs/2506.18497v1
• Date: Mon, 23 Jun 2025 10:49:19 GMT
• Title: Leveraging neural network interatomic potentials for a foundation model of chemistry
• Authors: So Yeon Kim, Yang Jeong Park, Ju Li,
• Abstract summary: HackNIP is a two-stage pipeline that leverages pretrained neural network interatomic potentials.<n>It first extracts fixed-length feature vectors from NIP foundation models and then uses these embeddings to train shallow ML models.<n>This study investigates whether such a hybridization approach, by hacking" the NIP, can outperform end-to-end deep neural networks.
• Score: 2.66269503676104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale foundation models, including neural network interatomic potentials (NIPs) in computational materials science, have demonstrated significant potential. However, despite their success in accelerating atomistic simulations, NIPs face challenges in directly predicting electronic properties and often require coupling to higher-scale models or extensive simulations for macroscopic properties. Machine learning (ML) offers alternatives for structure-to-property mapping but faces trade-offs: feature-based methods often lack generalizability, while deep neural networks require significant data and computational power. To address these trade-offs, we introduce HackNIP, a two-stage pipeline that leverages pretrained NIPs. This method first extracts fixed-length feature vectors (embeddings) from NIP foundation models and then uses these embeddings to train shallow ML models for downstream structure-to-property predictions. This study investigates whether such a hybridization approach, by ``hacking" the NIP, can outperform end-to-end deep neural networks, determines the dataset size at which this transfer learning approach surpasses direct fine-tuning of the NIP, and identifies which NIP embedding depths yield the most informative features. HackNIP is benchmarked on Matbench, evaluated for data efficiency, and tested on diverse tasks including \textit{ab initio}, experimental, and molecular properties. We also analyze how embedding depth impacts performance. This work demonstrates a hybridization strategy to overcome ML trade-offs in materials science, aiming to democratize high-performance predictive modeling.

Related papers

• Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks [45.752780408098765]
  Physics-informed neural networks (PINNs) offer a powerful approach to solving partial differential equations (PDEs)<n>PINNs have recently emerged as a valuable tool in the field of scientific machine learning.
  arXiv  Detail & Related papers  (2025-06-19T16:21:14Z)
• Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization [66.03821840425539]
  In this paper, we investigate the training dynamics of $L$-layer neural networks using the tensor gradient program (SGD) framework.<n>We show that SGD enables these networks to learn linearly independent features that substantially deviate from their initial values.<n>This rich feature space captures relevant data information and ensures that any convergent point of the training process is a global minimum.
  arXiv  Detail & Related papers  (2025-03-12T17:33:13Z)
• Multiscale Analysis of Woven Composites Using Hierarchical Physically Recurrent Neural Networks [0.0]
  Multiscale homogenization of woven composites requires detailed micromechanical evaluations.<n>This study introduces a Hierarchical Physically Recurrent Neural Network (HPRNN) employing two levels of surrogate modeling.
  arXiv  Detail & Related papers  (2025-03-06T19:02:32Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains [4.340917737559795]
  We study scaling in Neural Network Interatomic Potentials (NNIPs) NNIPs act as surrogate models for ab initio quantum mechanical calculations. We develop an NNIP architecture designed for scaling: the Efficiently Scaled Attention Interatomic Potential (EScAIP)
  arXiv  Detail & Related papers  (2024-10-31T17:35:57Z)
• Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks [7.956678963695681]
  We explore intersections between sparse coding and deep learning to enhance our understanding of feature extraction capabilities.<n>We derive convergence rates for convolutional neural networks (CNNs) in their ability to extract sparse features.<n>Inspired by the strong connection between sparse coding and CNNs, we explore training strategies to encourage neural networks to learn more sparse features.
  arXiv  Detail & Related papers  (2024-08-10T12:43:55Z)
• Physics-Informed Neural Networks with Hard Linear Equality Constraints [9.101849365688905]
  This work proposes a novel physics-informed neural network, KKT-hPINN, which rigorously guarantees hard linear equality constraints. Experiments on Aspen models of a stirred-tank reactor unit, an extractive distillation subsystem, and a chemical plant demonstrate that this model can further enhance the prediction accuracy.
  arXiv  Detail & Related papers  (2024-02-11T17:40:26Z)
• Spline-based neural network interatomic potentials: blending classical and machine learning models [0.0]
  We introduce a new MLIP framework which blends the simplicity of spline-based MEAM potentials with the flexibility of a neural network architecture. We demonstrate how this framework can be used to probe the boundary between classical and ML IPs.
  arXiv  Detail & Related papers  (2023-10-04T15:42:26Z)
• Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression [37.69303106863453]
  The Information Bottleneck (IB) principle offers an information-theoretic framework for analyzing the training process of deep neural networks (DNNs) In this paper, we introduce a framework for conducting IB analysis of general NNs. We also perform IB analysis on a close-to-real-scale, which reveals new features of the MI dynamics.
  arXiv  Detail & Related papers  (2023-05-13T21:44:32Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.