A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention

• URL: http://arxiv.org/abs/2603.06567v1
• Date: Fri, 06 Mar 2026 18:57:04 GMT
• Title: A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention
• Abstract summary: AllScAIP is an attention-based machine-learning model that scales to O(100 million) training samples.<n>It addresses the long-range challenge using an all-to-all node attention component that is data-driven.<n>Our model achieves state-of-the-art energy/force accuracy on molecular systems, as well as a number of physics-based evaluations.
• Score: 14.94564172199991
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine-learning interatomic potentials (MLIPs) have advanced rapidly, with many top models relying on strong physics-based inductive biases. However, as models scale to larger systems like biomolecules and electrolytes, they struggle to accurately capture long-range (LR) interactions, leading current approaches to rely on explicit physics-based terms or components. In this work, we propose AllScAIP, a straightforward, attention-based, and energy-conserving MLIP model that scales to O(100 million) training samples. It addresses the long-range challenge using an all-to-all node attention component that is data-driven. Extensive ablations reveal that in low-data/small-model regimes, inductive biases improve sample efficiency. However, as data and model size scale, these benefits diminish or even reverse, while all-to-all attention remains critical for capturing LR interactions. Our model achieves state-of-the-art energy/force accuracy on molecular systems, as well as a number of physics-based evaluations (OMol25), while being competitive on materials (OMat24) and catalysts (OC20). Furthermore, it enables stable, long-timescale MD simulations that accurately recover experimental observables, including density and heat of vaporization predictions.

Related papers

• How accurate are foundational machine learning interatomic potentials for heterogeneous catalysis? [0.0]
  Foundational machine learning interatomic potentials (MLIPs) are being developed at a rapid pace, promising closer and closer approximation to ab initio accuracy.<n>Here, we systematically analyze zero-shot performance of 80 different MLIPs, evaluating tasks typical for heterogeneous tasks.<n>We find that many MLIPs catastrophically fail when applied to magnetic materials, and structure relaxation in the MLIP generally increases the energy prediction error.
  arXiv  Detail & Related papers  (2025-12-18T16:06:14Z)
• A universal augmentation framework for long-range electrostatics in machine learning interatomic potentials [8.414405667524889]
  Latent Ewald Summation (LES) method infers electrostatic interactions, polarization, and Born effective charges (BECs)<n>We present LES as a standalone library, compatible with any short-range MLIP, and demonstrate its integration with methods such as MACE, NequIP, CACE, and CHGNet.<n>We scale LES to large and chemically diverse data by training MACELES-OFF on the SPICE set containing molecules and clusters, making a universal MLIP with electrostatics for organic systems including biomolecules.
  arXiv  Detail & Related papers  (2025-07-18T18:21:45Z)
• Accurate Ab-initio Neural-network Solutions to Large-Scale Electronic Structure Problems [52.19558333652367]
  We present finite-range embeddings (FiRE) for accurate large-scale ab-initio electronic structure calculations.<n>FiRE reduces the complexity of neural-network variational Monte Carlo (NN-VMC) by $sim ntextel$, the number of electrons.<n>We validate our method's accuracy on various challenging systems, including biochemical compounds and organometallic compounds.
  arXiv  Detail & Related papers  (2025-04-08T14:28:54Z)
• Predicting ionic conductivity in solids from the machine-learned potential energy landscape [68.25662704255433]
  We propose an approach for the quick and reliable screening of ionic conductors through the analysis of a universal interatomic potential.<n>Eight out of the ten highest-ranked materials are confirmed to be superionic at room temperature in first-principles calculations.<n>Our method achieves a speed-up factor of approximately 50 compared to molecular dynamics driven by a machine-learning potential, and is at least 3,000 times faster compared to first-principles molecular dynamics.
  arXiv  Detail & Related papers  (2024-11-11T09:01:36Z)
• A Comprehensive Evaluation of Generative Models in Calorimeter Shower Simulation [0.0]
  "Fast Simulation" has been pivotal in overcoming computational bottlenecks. The use of deep-generative models has sparked a surge of interest in surrogate modeling for detector simulations. Our evaluation revealed that the CaloDiffusion and CaloScore generative models demonstrate the most accurate simulation of particle showers.
  arXiv  Detail & Related papers  (2024-06-08T11:17:28Z)
• A conditional latent autoregressive recurrent model for generation and forecasting of beam dynamics in particle accelerators [46.348283638884425]
  We propose a two-step unsupervised deep learning framework named as Latent Autoregressive Recurrent Model (CLARM) for learning dynamics of charged particles in accelerators. The CLARM can generate projections at various accelerator sampling modules by capturing and decoding the latent space representation. The results demonstrate that the generative and forecasting ability of the proposed approach is promising when tested against a variety of evaluation metrics.
  arXiv  Detail & Related papers  (2024-03-19T22:05:17Z)
• Conditional Generative Models for Simulation of EMG During Naturalistic Movements [45.698312905115955]
  We present a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy.
  arXiv  Detail & Related papers  (2022-11-03T14:49:02Z)
• Electronic-structure properties from atom-centered predictions of the electron density [0.0]
  electron density of a molecule or material has recently received major attention as a target quantity of machine-learning models. We propose a gradient-based approach to minimize the loss function of the regression problem in an optimized and highly sparse feature space. We show that starting from the predicted density a single Kohn-Sham diagonalization step can be performed to access total energy components that carry an error of just 0.1 meV/atom.
  arXiv  Detail & Related papers  (2022-06-28T15:35:55Z)
• Physics-informed CoKriging model of a redox flow battery [68.8204255655161]
  Redox flow batteries (RFBs) offer the capability to store large amounts of energy cheaply and efficiently. There is a need for fast and accurate models of the charge-discharge curve of a RFB to potentially improve the battery capacity and performance. We develop a multifidelity model for predicting the charge-discharge curve of a RFB.
  arXiv  Detail & Related papers  (2021-06-17T00:49:55Z)
• BIGDML: Towards Exact Machine Learning Force Fields for Materials [55.944221055171276]
  Machine-learning force fields (MLFF) should be accurate, computationally and data efficient, and applicable to molecules, materials, and interfaces thereof. Here, we introduce the Bravais-Inspired Gradient-Domain Machine Learning approach and demonstrate its ability to construct reliable force fields using a training set with just 10-200 atoms.
  arXiv  Detail & Related papers  (2021-06-08T10:14:57Z)
• Predicting molecular dipole moments by combining atomic partial charges and atomic dipoles [3.0980025155565376]
  "MuML" models are fitted together to reproduce molecular $boldsymbolmu$ computed using high-level coupled-cluster theory. We demonstrate that the uncertainty in the predictions can be estimated reliably using a calibrated committee model.
  arXiv  Detail & Related papers  (2020-03-27T14:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.