Teacher-student training improves accuracy and efficiency of machine learning inter-atomic potentials

• URL: http://arxiv.org/abs/2502.05379v1
• Date: Fri, 07 Feb 2025 23:20:43 GMT
• Title: Teacher-student training improves accuracy and efficiency of machine learning inter-atomic potentials
• Authors: Sakib Matin, Alice Allen, Emily Shinkle, Aleksandra Pachalieva, Galen T. Craven, Benjamin Nebgen, Justin Smith, Richard Messerly, Ying Wai Li, Sergei Tretiak, Kipton Barros, Nicholas Lubbers,
• Abstract summary: Machine learning inter-atomic potentials (MLIPs) are revolutionizing the field of molecular dynamics (MD) simulations. We present a teacher-student training framework in which the latent knowledge from the teacher (atomic energies) is used to augment the students' training. Remarkably, the student models can even surpass the accuracy of the teachers, even though both are trained on the same quantum chemistry dataset.
• Score: 31.114245664719455
• License:
• Abstract: Machine learning inter-atomic potentials (MLIPs) are revolutionizing the field of molecular dynamics (MD) simulations. Recent MLIPs have tended towards more complex architectures trained on larger datasets. The resulting increase in computational and memory costs may prohibit the application of these MLIPs to perform large-scale MD simulations. Here, we present a teacher-student training framework in which the latent knowledge from the teacher (atomic energies) is used to augment the students' training. We show that the light-weight student MLIPs have faster MD speeds at a fraction of the memory footprint compared to the teacher models. Remarkably, the student models can even surpass the accuracy of the teachers, even though both are trained on the same quantum chemistry dataset. Our work highlights a practical method for MLIPs to reduce the resources required for large-scale MD simulations.

Related papers

• Energy & Force Regression on DFT Trajectories is Not Enough for Universal Machine Learning Interatomic Potentials [8.254607304215451]
  Universal Machine Learning Interactomic Potentials (MLIPs) enable accelerated simulations for materials discovery. MLIPs' inability to reliably and accurately perform large-scale molecular dynamics (MD) simulations for diverse materials.
  arXiv  Detail & Related papers  (2025-02-05T23:04:21Z)
• BoostMD: Accelerating molecular sampling by leveraging ML force field features from previous time-steps [3.8214695776749013]
  BoostMD is a surrogate model architecture designed to accelerate molecular dynamics simulations. Our experiments demonstrate that BoostMD achieves an eight-fold speedup compared to the reference model. By combining efficient feature reuse with a streamlined architecture, BoostMD offers a robust solution for conducting large-scale, long-timescale molecular simulations.
  arXiv  Detail & Related papers  (2024-12-21T20:52:36Z)
• MiniPLM: Knowledge Distillation for Pre-Training Language Models [109.83741809808483]
  MiniPLM is a KD framework for pre-training student language models. For efficiency, MiniPLM performs offline teacher LM inference, allowing KD for multiple student LMs without adding training-time costs. For flexibility, MiniPLM operates solely on the training corpus, enabling KD across model families.
  arXiv  Detail & Related papers  (2024-10-22T17:40:32Z)
• On Optimizing Hyperparameters for Quantum Neural Networks [0.5999777817331317]
  Current state-of-the-art Machine Learning models require weeks for training, which is associated with an enormous $CO$ footprint. Quantum Computing, and specifically Quantum Machine Learning (QML), can offer significant theoretical speed-ups and enhanced power.
  arXiv  Detail & Related papers  (2024-03-27T13:59:09Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [73.35846234413611]
  In drug discovery, molecular dynamics (MD) simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning (ML) surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding dynamics. We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$times$ speedup compared to standard numerical MD simulations.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD [51.04126395480625]
  Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. This work offers a solution by simplifying this coupling and enabling in situ training and inference on heterogeneous clusters.
  arXiv  Detail & Related papers  (2023-06-22T14:07:54Z)
• Bulk-Switching Memristor-based Compute-In-Memory Module for Deep Neural Network Training [15.660697326769686]
  We propose a mixed-precision training scheme for memristor-based compute-in-memory (CIM) modules. The proposed scheme is implemented with a system-on-chip (SoC) of fully integrated analog CIM modules and digital sub-systems. The efficacy of training larger models is evaluated using realistic hardware parameters and shows that analog CIM modules can enable efficient mix-precision training with accuracy comparable to full-precision software trained models.
  arXiv  Detail & Related papers  (2023-05-23T22:03:08Z)
• Continual learning autoencoder training for a particle-in-cell simulation via streaming [52.77024349608834]
  upcoming exascale era will provide a new generation of physics simulations with high resolution. These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
  arXiv  Detail & Related papers  (2022-11-09T09:55:14Z)
• SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering [49.78647219715034]
  We propose a sensing-aware model-based reinforcement learning system called SAM-RL. With the sensing-aware learning pipeline, SAM-RL allows a robot to select an informative viewpoint to monitor the task process. We apply our framework to real world experiments for accomplishing three manipulation tasks: robotic assembly, tool manipulation, and deformable object manipulation.
  arXiv  Detail & Related papers  (2022-10-27T05:30:43Z)
• Machine Learning Training on a Real Processing-in-Memory System [9.286176889576996]
  Training machine learning algorithms is a computationally intensive process, which is frequently memory-bound. Memory-centric computing systems with processing-in-memory capabilities can alleviate this data movement bottleneck. Our work is the first one to evaluate training of machine learning algorithms on a real-world general-purpose PIM architecture.
  arXiv  Detail & Related papers  (2022-06-13T10:20:23Z)
• Accurate Machine Learned Quantum-Mechanical Force Fields for Biomolecular Simulations [51.68332623405432]
  Molecular dynamics (MD) simulations allow atomistic insights into chemical and biological processes. Recently, machine learned force fields (MLFFs) emerged as an alternative means to execute MD simulations. This work proposes a general approach to constructing accurate MLFFs for large-scale molecular simulations.
  arXiv  Detail & Related papers  (2022-05-17T13:08:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.