Enhancing Machine Learning Potentials through Transfer Learning across Chemical Elements

• URL: http://arxiv.org/abs/2502.13522v1
• Date: Wed, 19 Feb 2025 08:20:54 GMT
• Title: Enhancing Machine Learning Potentials through Transfer Learning across Chemical Elements
• Authors: Sebastien Röcken, Julija Zavadlav,
• Abstract summary: Machine Learning Potentials (MLPs) can enable simulations of ab initio accuracy at orders of magnitude lower computational cost.<n>Here, we introduce transfer learning of potential energy surfaces between chemically similar elements.<n>We demonstrate that transfer learning surpasses traditional training from scratch in force prediction, leading to more stable simulations and improved temperature transferability.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine Learning Potentials (MLPs) can enable simulations of ab initio accuracy at orders of magnitude lower computational cost. However, their effectiveness hinges on the availability of considerable datasets to ensure robust generalization across chemical space and thermodynamic conditions. The generation of such datasets can be labor-intensive, highlighting the need for innovative methods to train MLPs in data-scarce scenarios. Here, we introduce transfer learning of potential energy surfaces between chemically similar elements. Specifically, we leverage the trained MLP for silicon to initialize and expedite the training of an MLP for germanium. Utilizing classical force field and ab initio datasets, we demonstrate that transfer learning surpasses traditional training from scratch in force prediction, leading to more stable simulations and improved temperature transferability. These advantages become even more pronounced as the training dataset size decreases. The out-of-target property analysis shows that transfer learning leads to beneficial but sometimes adversarial effects. Our findings demonstrate that transfer learning across chemical elements is a promising technique for developing accurate and numerically stable MLPs, particularly in a data-scarce regime.

Related papers

• ReMem: Mutual Information-Aware Fine-tuning of Pretrained Vision Transformers for Effective Knowledge Distillation [55.55242848676581]
  Knowledge distillation from pretrained visual representation models offers an effective approach to improve small, task-specific production models.<n>However, the effectiveness of such knowledge transfer drops significantly when distilling from strong models that are pretrained in a large scale.<n>Motivated by the connection between mutual information and distillation effectiveness, we propose to employ mutual information-aware optimization during finetuning.
  arXiv  Detail & Related papers  (2025-06-29T00:25:23Z)
• Ensemble Knowledge Distillation for Machine Learning Interatomic Potentials [34.82692226532414]
  We present an ensemble knowledge distillation (EKD) method to improve machine learning interatomic potentials (MLIPs)<n>First, multiple teacher models are trained to QC energies and then generate atomic forces for all configurations in the dataset. Next, the student MLIP is trained to both QC energies and to ensemble-averaged forces generated by the teacher models.<n>The resulting student MLIPs achieve new state-of-the-art accuracy on the COMP6 benchmark and show improved stability for molecular dynamics simulations.
  arXiv  Detail & Related papers  (2025-03-18T14:32:51Z)
• Non-asymptotic Convergence of Training Transformers for Next-token Prediction [48.9399496805422]
  Transformers have achieved extraordinary success in modern machine learning due to their excellent ability to handle sequential data. This paper provides a fine-grained non-asymptotic analysis of the training dynamics of a one-layer transformer. We show that the trained transformer presents non-token prediction ability with dataset shift.
  arXiv  Detail & Related papers  (2024-09-25T20:22:06Z)
• Physics-Informed Weakly Supervised Learning for Interatomic Potentials [17.165117198519248]
  We introduce a physics-informed, weakly supervised approach for training machine-learned interatomic potentials. We demonstrate reduced energy and force errors -- often lower by a factor of two -- for various baseline models and benchmark data sets.
  arXiv  Detail & Related papers  (2024-07-23T12:49:04Z)
• Optimal design of experiments in the context of machine-learning inter-atomic potentials: improving the efficiency and transferability of kernel based methods [0.7234862895932991]
  Data-driven, machine learning (ML) models of atomistic interactions can relate nuanced aspects of atomic arrangements into predictions of energies and forces. The main challenge stems from the fact that descriptors of chemical environments are often sparse high-dimensional objects without a well-defined continuous metric. We will demonstrate that classical concepts of statistical planning of experiments and optimal design can help to mitigate such problems at a relatively low computational cost.
  arXiv  Detail & Related papers  (2024-05-14T14:14:23Z)
• Considerations in the use of ML interaction potentials for free energy calculations [0.0]
  Machine learning potentials (MLPs) offer the potential to accurately model the energy and free energy landscapes of molecules. We examined how the distribution of collective variables (CVs) in the training data affects accuracy in determining the free energy surface (FES) of systems. Findings for butane revealed that training data coverage of key FES regions ensures model accuracy regardless of CV distribution.
  arXiv  Detail & Related papers  (2024-03-20T19:49:21Z)
• Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes [57.62036621319563]
  We introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. We demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators.
  arXiv  Detail & Related papers  (2023-12-19T12:34:46Z)
• Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly [62.473245910234304]
  This paper takes a hardware-centric approach to explore how Large Language Models can be brought to modern edge computing systems. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions.
  arXiv  Detail & Related papers  (2023-10-04T20:27:20Z)
• Lifelong Machine Learning Potentials [0.0]
  We introduce element-embracing atom-centered functions (eeACSFs) which combine structural properties and element information from the periodic table. We apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data.
  arXiv  Detail & Related papers  (2023-03-10T13:38:36Z)
• Transfer learning for chemically accurate interatomic neural network potentials [0.0]
  We show that pre-training the network parameters on data obtained from density functional calculations improves the sample efficiency of models trained on more accurate ab-initio data. We provide GM-NN potentials pre-trained and fine-tuned on the ANI-1x and ANI-1ccx data sets, which can easily be fine-tuned on and applied to organic molecules.
  arXiv  Detail & Related papers  (2022-12-07T19:21:01Z)
• Improving Molecular Representation Learning with Metric Learning-enhanced Optimal Transport [49.237577649802034]
  We develop a novel optimal transport-based algorithm termed MROT to enhance their generalization capability for molecular regression problems. MROT significantly outperforms state-of-the-art models, showing promising potential in accelerating the discovery of new substances.
  arXiv  Detail & Related papers  (2022-02-13T04:56:18Z)
• Federated Learning of Molecular Properties in a Heterogeneous Setting [79.00211946597845]
  We introduce federated heterogeneous molecular learning to address these challenges. Federated learning allows end-users to build a global model collaboratively while preserving the training data distributed over isolated clients. FedChem should enable a new type of collaboration for improving AI in chemistry that mitigates concerns about valuable chemical data.
  arXiv  Detail & Related papers  (2021-09-15T12:49:13Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)
• Machine Learning Force Fields [54.48599172620472]
  Machine Learning (ML) has enabled numerous advances in computational chemistry. One of the most promising applications is the construction of ML-based force fields (FFs) This review gives an overview of applications of ML-FFs and the chemical insights that can be obtained from them.
  arXiv  Detail & Related papers  (2020-10-14T13:14:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.