Knowledge Distillation Framework for Accelerating High-Accuracy Neural Network-Based Molecular Dynamics Simulations

• URL: http://arxiv.org/abs/2506.15337v2
• Date: Fri, 20 Jun 2025 05:31:52 GMT
• Title: Knowledge Distillation Framework for Accelerating High-Accuracy Neural Network-Based Molecular Dynamics Simulations
• Authors: Naoki Matsumura, Yuta Yoshimoto, Yuto Iwasaki, Meguru Yamazaki, Yasufumi Sakai,
• Abstract summary: We propose a novel KD framework that leverages a non-fine-tuned, off-the-shelf pre-trained NNP as a teacher.<n>Our framework employs a two-stage training process: first, the student NNP is trained with a dataset generated by the off-the-shelf teacher; then, it is fine-tuned with a smaller, high-accuracy density functional theory (DFT) dataset.<n>We demonstrate the effectiveness of our framework by applying it to both organic (polyethylene glycol) and inorganic (L$_10$GeP$_2$S$_
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural network potentials (NNPs) offer a powerful alternative to traditional force fields for molecular dynamics (MD) simulations. Accurate and stable MD simulations, crucial for evaluating material properties, require training data encompassing both low-energy stable structures and high-energy structures. Conventional knowledge distillation (KD) methods fine-tune a pre-trained NNP as a teacher model to generate training data for a student model. However, in material-specific models, this fine-tuning process increases energy barriers, making it difficult to create training data containing high-energy structures. To address this, we propose a novel KD framework that leverages a non-fine-tuned, off-the-shelf pre-trained NNP as a teacher. Its gentler energy landscape facilitates the exploration of a wider range of structures, including the high-energy structures crucial for stable MD simulations. Our framework employs a two-stage training process: first, the student NNP is trained with a dataset generated by the off-the-shelf teacher; then, it is fine-tuned with a smaller, high-accuracy density functional theory (DFT) dataset. We demonstrate the effectiveness of our framework by applying it to both organic (polyethylene glycol) and inorganic (L$_{10}$GeP$_{2}$S$_{12}$) materials, achieving comparable or superior accuracy in reproducing physical properties compared to existing methods. Importantly, our method reduces the number of expensive DFT calculations by 10x compared to existing NNP generation methods, without sacrificing accuracy. Furthermore, the resulting student NNP achieves up to 106x speedup in inference compared to the teacher NNP, enabling significantly faster and more efficient MD simulations.

Related papers

• LaMM: Semi-Supervised Pre-Training of Large-Scale Materials Models [0.1999925939110439]
  We propose LaMM, a semi-supervised pre-training method incorporating improved denoising self-supervised learning and a load-balancing algorithm for efficient multi-node training.<n>We demonstrate that our approach effectively leverages a large-scale dataset of $sim$300 million semi-labeled samples to train a single NNP model, resulting in improved fine-tuning performance in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2025-05-28T10:36:49Z)
• On the Connection Between Diffusion Models and Molecular Dynamics [0.0]
  Denoising diffusion models have shown promise in NNPs by training networks to remove noise added to stable configurations.<n>We show how a denoising model can be implemented using a conventional MD software package interfaced with a standard NNP architecture.
  arXiv  Detail & Related papers  (2025-04-04T05:32:38Z)
• Ensemble Knowledge Distillation for Machine Learning Interatomic Potentials [34.82692226532414]
  We present an ensemble knowledge distillation (EKD) method to improve machine learning interatomic potentials (MLIPs)<n>First, multiple teacher models are trained to QC energies and then generate atomic forces for all configurations in the dataset. Next, the student MLIP is trained to both QC energies and to ensemble-averaged forces generated by the teacher models.<n>The resulting student MLIPs achieve new state-of-the-art accuracy on the COMP6 benchmark and show improved stability for molecular dynamics simulations.
  arXiv  Detail & Related papers  (2025-03-18T14:32:51Z)
• A General Neural Network Potential for Energetic Materials with C, H, N, and O elements [0.9742644628669695]
  High-energy materials (HEMs) are constrained by the prohibitive computational expense and prolonged development cycles.<n>We develop a general neural network potential (NNP) that efficiently predicts the structural, mechanical, and decomposition properties of HEMs.
  arXiv  Detail & Related papers  (2025-03-03T03:24:59Z)
• Implicit Delta Learning of High Fidelity Neural Network Potentials [1.135672229709142]
  Implicit Delta Learning (IDLe) method reduces the need for high-fidelity Quantum Mechanics (QM) data.<n>IDLe achieves the same accuracy as single high-fidelity baselines while using up to 50x less high-fidelity data.
  arXiv  Detail & Related papers  (2024-12-08T20:35:45Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [73.35846234413611]
  In drug discovery, molecular dynamics (MD) simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning (ML) surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding dynamics. We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$times$ speedup compared to standard numerical MD simulations.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• Transfer learning for chemically accurate interatomic neural network potentials [0.0]
  We show that pre-training the network parameters on data obtained from density functional calculations improves the sample efficiency of models trained on more accurate ab-initio data. We provide GM-NN potentials pre-trained and fine-tuned on the ANI-1x and ANI-1ccx data sets, which can easily be fine-tuned on and applied to organic molecules.
  arXiv  Detail & Related papers  (2022-12-07T19:21:01Z)
• A Kernel-Based View of Language Model Fine-Tuning [94.75146965041131]
  We investigate whether the Neural Tangent Kernel (NTK) describes fine-tuning of pre-trained LMs. We show that formulating the downstream task as a masked word prediction problem through prompting often induces kernel-based dynamics during fine-tuning.
  arXiv  Detail & Related papers  (2022-10-11T17:34:32Z)
• Multi-fidelity Hierarchical Neural Processes [79.0284780825048]
  Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling. We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2022-06-10T04:54:13Z)
• Accurate Machine Learned Quantum-Mechanical Force Fields for Biomolecular Simulations [51.68332623405432]
  Molecular dynamics (MD) simulations allow atomistic insights into chemical and biological processes. Recently, machine learned force fields (MLFFs) emerged as an alternative means to execute MD simulations. This work proposes a general approach to constructing accurate MLFFs for large-scale molecular simulations.
  arXiv  Detail & Related papers  (2022-05-17T13:08:28Z)
• Enhanced physics-constrained deep neural networks for modeling vanadium redox flow battery [62.997667081978825]
  We propose an enhanced version of the physics-constrained deep neural network (PCDNN) approach to provide high-accuracy voltage predictions. The ePCDNN can accurately capture the voltage response throughout the charge--discharge cycle, including the tail region of the voltage discharge curve.
  arXiv  Detail & Related papers  (2022-03-03T19:56:24Z)
• BIGDML: Towards Exact Machine Learning Force Fields for Materials [55.944221055171276]
  Machine-learning force fields (MLFF) should be accurate, computationally and data efficient, and applicable to molecules, materials, and interfaces thereof. Here, we introduce the Bravais-Inspired Gradient-Domain Machine Learning approach and demonstrate its ability to construct reliable force fields using a training set with just 10-200 atoms.
  arXiv  Detail & Related papers  (2021-06-08T10:14:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.