Overcoming systematic softening in universal machine learning interatomic potentials by fine-tuning

• URL: http://arxiv.org/abs/2405.07105v1
• Date: Sat, 11 May 2024 22:30:47 GMT
• Title: Overcoming systematic softening in universal machine learning interatomic potentials by fine-tuning
• Authors: Bowen Deng, Yunyeong Choi, Peichen Zhong, Janosh Riebesell, Shashwat Anand, Zhuohan Li, KyuJung Jun, Kristin A. Persson, Gerbrand Ceder,
• Abstract summary: Machine learning interatomic potentials (MLIPs) have introduced a new paradigm for atomic simulations. Recent advancements have seen the emergence of universal MLIPs (uMLIPs) that are pre-trained on diverse materials datasets. We show that their performance in extrapolating to out-of-distribution complex atomic environments remains unclear.
• Score: 3.321322648845526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning interatomic potentials (MLIPs) have introduced a new paradigm for atomic simulations. Recent advancements have seen the emergence of universal MLIPs (uMLIPs) that are pre-trained on diverse materials datasets, providing opportunities for both ready-to-use universal force fields and robust foundations for downstream machine learning refinements. However, their performance in extrapolating to out-of-distribution complex atomic environments remains unclear. In this study, we highlight a consistent potential energy surface (PES) softening effect in three uMLIPs: M3GNet, CHGNet, and MACE-MP-0, which is characterized by energy and force under-prediction in a series of atomic-modeling benchmarks including surfaces, defects, solid-solution energetics, phonon vibration modes, ion migration barriers, and general high-energy states. We find that the PES softening behavior originates from a systematic underprediction error of the PES curvature, which derives from the biased sampling of near-equilibrium atomic arrangements in uMLIP pre-training datasets. We demonstrate that the PES softening issue can be effectively rectified by fine-tuning with a single additional data point. Our findings suggest that a considerable fraction of uMLIP errors are highly systematic, and can therefore be efficiently corrected. This result rationalizes the data-efficient fine-tuning performance boost commonly observed with foundational MLIPs. We argue for the importance of a comprehensive materials dataset with improved PES sampling for next-generation foundational MLIPs.

Related papers

• DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs [70.91804882618243]
  This paper proposes DSMoE, a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks. We implement adaptive expert routing using sigmoid activation and straight-through estimators, enabling tokens to flexibly access different aspects of model knowledge. Experiments on LLaMA models demonstrate that under equivalent computational constraints, DSMoE achieves superior performance compared to existing pruning and MoE approaches.
  arXiv  Detail & Related papers  (2025-02-18T02:37:26Z)
• Feasible Learning [78.6167929413604]
  We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample. Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.
  arXiv  Detail & Related papers  (2025-01-24T20:39:38Z)
• MAPS: Advancing Multi-Modal Reasoning in Expert-Level Physical Science [62.96434290874878]
  Current Multi-Modal Large Language Models (MLLM) have shown strong capabilities in general visual reasoning tasks. We develop a new framework, named Multi-Modal Scientific Reasoning with Physics Perception and Simulation (MAPS) based on an MLLM. MAPS decomposes expert-level multi-modal reasoning task into physical diagram understanding via a Physical Perception Model (PPM) and reasoning with physical knowledge via a simulator.
  arXiv  Detail & Related papers  (2025-01-18T13:54:00Z)
• Materials Learning Algorithms (MALA): Scalable Machine Learning for Electronic Structure Calculations in Large-Scale Atomistic Simulations [2.04071520659173]
  We present the Materials Learning Algorithms (MALA) package, a scalable machine learning framework suitable for large-scale atomistic simulations. MALA models efficiently predict key electronic observables, including local density of states, electronic density, density of states, and total energy. We demonstrate MALA's capabilities with examples including boron clusters, aluminum across its solid-liquid phase boundary, and predicting the electronic structure of a stacking fault in a large beryllium slab.
  arXiv  Detail & Related papers  (2024-11-29T11:10:29Z)
• Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning [104.27224674122313]
  Fine-tuning MLLM has become a common practice to improve performance on specific downstream tasks. To balance the trade-off between generalization and specialization, we propose measuring the parameter importance for both pre-trained and fine-tuning distributions.
  arXiv  Detail & Related papers  (2024-11-17T01:16:37Z)
• EPi-cKANs: Elasto-Plasticity Informed Kolmogorov-Arnold Networks Using Chebyshev Polynomials [0.0]
  We present an elasto-plasticity informed Chebyshev-based network (EPi-cKAN) EPi-cKAN provides superior accuracy in predicting stress components and demonstrates better accuracy when used to predict sand elasto-plastic behavior under blind triaxial axisymmetric strain-controlled loading paths.
  arXiv  Detail & Related papers  (2024-10-12T16:01:38Z)
• Interpolation and differentiation of alchemical degrees of freedom in machine learning interatomic potentials [0.980222898148295]
  We report the use of continuous and differentiable alchemical degrees of freedom in atomistic materials simulations. The proposed method introduces alchemical atoms with corresponding weights into the input graph, alongside modifications to the message-passing and readout mechanisms of MLIPs. The end-to-end differentiability of MLIPs enables efficient calculation of the gradient of energy with respect to the compositional weights.
  arXiv  Detail & Related papers  (2024-04-16T17:24:22Z)
• EL-MLFFs: Ensemble Learning of Machine Leaning Force Fields [1.8367772188990783]
  Machine learning force fields (MLFFs) have emerged as a promising approach to bridge the accuracy of quantum mechanical methods. We propose a novel ensemble learning framework, EL-MLFFs, which leverages the stacking method to integrate predictions from diverse MLFFs. We evaluate our approach on two distinct datasets: methane molecules and methanol adsorbed on a Cu(100) surface.
  arXiv  Detail & Related papers  (2024-03-26T09:09:40Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• Pre-training via Denoising for Molecular Property Prediction [53.409242538744444]
  We describe a pre-training technique that utilizes large datasets of 3D molecular structures at equilibrium. Inspired by recent advances in noise regularization, our pre-training objective is based on denoising.
  arXiv  Detail & Related papers  (2022-05-31T22:28:34Z)
• Automated discovery of a robust interatomic potential for aluminum [4.6028828826414925]
  Machine learning (ML) based potentials aim for faithful emulation of quantum mechanics (QM) calculations at drastically reduced computational cost. We present a highly automated approach to dataset construction using the principles of active learning (AL) We demonstrate this approach by building an ML potential for aluminum (ANI-Al) To demonstrate transferability, we perform a 1.3M atom shock simulation, and show that ANI-Al predictions agree very well with DFT calculations on local atomic environments sampled from the nonequilibrium dynamics.
  arXiv  Detail & Related papers  (2020-03-10T19:06:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.