Pretrained Model Representations as Acquisition Signals for Active Learning of MLIPs

• URL: http://arxiv.org/abs/2605.03964v1
• Date: Tue, 05 May 2026 16:48:23 GMT
• Title: Pretrained Model Representations as Acquisition Signals for Active Learning of MLIPs
• Abstract summary: We investigate whether the latent space of a pretrained MLIP already contains the information necessary for effective acquisition.<n>We introduce two acquisition signals derived directly from a pretrained MACE potential.<n>Our results suggest that pretraining aligns latent-space geometry with model error, yielding a practical and sufficient acquisition signal for reactive MLIP fine-tuning.
• Score: 4.4406458484306555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training machine learning interatomic potentials (MLIPs) for reactive chemistry is often bottlenecked by the high cost of quantum chemical labels and the scarcity of transition state configurations in candidate pools. Active learning (AL) can mitigate these costs, but its effectiveness hinges on the acquisition rule. We investigate whether the latent space of a pretrained MLIP already contains the information necessary for effective acquisition, eliminating the need for auxiliary uncertainty heads, Bayesian training and fine-tuning, or committee ensembles. We introduce two acquisition signals derived directly from a pretrained MACE potential: a finite-width neural tangent kernel (NTK) and an activation kernel built from hidden latent space features. On reactive-chemistry benchmarks, both kernels consistently outperform fixed-descriptor baselines, committee disagreement, and random acquisition, reducing the data required to reach performance targets by an average of 38% for energy error and 28% for force error. We further show that the pretrained model induces similarity spaces that preserve chemically meaningful structure and provide more reliable residual uncertainty estimates than randomly initialised or fixed-descriptor-based kernels. Our results suggest that pretraining aligns latent-space geometry with model error, yielding a practical and sufficient acquisition signal for reactive MLIP fine-tuning.

Related papers

• AdaptNTK: Adaptive Uncertainty Quantification and Active Learning for Neural Network Potentials [0.0]
  We introduce AdaptNTK, a single-model framework that measures uncertainty as a regularized Mahalanobis distance in empirical neural tangent kernel feature space.<n>With the NTK features fixed during acquisition, the uncertainty depends on the acquired configurations but not their reference labels.<n>On held-out rMD17 data, AdaptNTK achieves the highest mean correlations with force errors (Spearman 0.68, Pearson 0.71) and matches a three-member ensemble in error retention.
  arXiv  Detail & Related papers  (2026-08-31T23:34:29Z)
• Scalable Lindblad Noise Learning via Stochastic Tensor-Network Simulation [48.887627688666974]
  Learning dissipation rates in large-scale open quantum systems is a major obstacle for near-term quantum technologies.<n>Here, we propose a scalable noise-learning framework for Lindblad dissipation rates that combines a simulation method, the Jump Method (TJM)<n>We demonstrate the approach on two noise models in the Ising model, in which independent dissipation rates are learned for each site up to $N_mathrmsite=16$.<n>This combination of scalable numerics and rigorous theoretical guarantees positions TJM-based noise learning as a practical foundation for characterizing dissipation in large quantum devices.
  arXiv  Detail & Related papers  (2026-08-25T15:07:11Z)
• Agentic multi-fidelity learning of quasiparticle and excitonic properties [0.5219568203653523]
  Many-body GW-Bethe-Salpeter equation calculations are essential for accurate simulations of electronic structure and optical properties in low-dimensional nanomaterials.<n>We introduce an agent-guided multi-fidelity framework for correcting GW-Bethe-Salpeter excited-state landscapes in strained MoS2-WS2 bilayers.
  arXiv  Detail & Related papers  (2026-06-05T20:55:23Z)
• Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training [0.06157382820537719]
  Quantization-Aware Training (QAT) has emerged as a leading compression approach.<n>We propose Neuron-Level Mixed-Precision QAT (NMP-QAT)<n>NMP-QAT achieves superior compression-accuracy trade-offs over mixed-precision QAT baselines.
  arXiv  Detail & Related papers  (2026-05-24T13:02:42Z)
• Force-Aware Neural Tangent Kernels for Scalable and Robust Active Learning of MLIPs [4.4406458484306555]
  We introduce a linearly scaling acquisition framework based on chunked feature-space posterior-variance shortlisting.<n>We then extend the Neural Tangent Kernel (NTK) to a force-aware setting via mixed parameter-coordinate derivatives.<n>We demonstrate the effectiveness of the joint energy-force NTK on the OC20 dataset, where force-aware acquisition is crucial.
  arXiv  Detail & Related papers  (2026-05-13T17:08:37Z)
• daVinci-LLM:Towards the Science of Pretraining [58.6458452580545]
  daVinci-LLM combines industrial-scale resources with full research freedom to advance the science of pretraining.<n>We release complete data processing pipelines, full training processes, and systematic exploration results.
  arXiv  Detail & Related papers  (2026-03-28T07:00:49Z)
• Activation-Space Uncertainty Quantification for Pretrained Networks [2.001149416674759]
  We introduce Gaussian Process Activations (GAPA), a post-hoc method that shifts Bayesian modeling from weights to activations.<n>GAPA replaces standard nonlinearities with activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction.<n>To scale to modern architectures, we use a sparse variational inducing-point approximation over cached training activations, combined with local k-nearest-neighbor conditioning.
  arXiv  Detail & Related papers  (2026-02-16T17:17:08Z)
• RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training [59.493415006017635]
  Pre-trained Multi-modal Large Language Models (MLLMs) provide a knowledge-rich foundation for post-training.<n>Current evaluation relies on testing after supervised fine-tuning, which introduces laborious additional training and autoregressive decoding costs.<n>We propose RADAR, an efficient ability-centric evaluation framework for Revealing Asymmetric Development of Abilities in MLLM pRe-training.
  arXiv  Detail & Related papers  (2026-02-13T12:56:31Z)
• Equivariant Evidential Deep Learning for Interatomic Potentials [55.6997213490859]
  Uncertainty quantification is critical for assessing the reliability of machine learning interatomic potentials in molecular dynamics simulations.<n>Existing UQ approaches for MLIPs are often limited by high computational cost or suboptimal performance.<n>We propose textitEquivariant Evidential Deep Learning for Interatomic Potentials ($texte2$IP), a backbone-agnostic framework that models atomic forces and their uncertainty jointly.
  arXiv  Detail & Related papers  (2026-02-11T02:00:25Z)
• Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approach [35.66975292316149]
  Separating liquid-liquid dispersions in gravity settlers is critical in chemical, pharmaceutical, and recycling processes.<n>Dense-packed zone height is an important performance and safety indicator but it is often expensive and impractical to measure.<n>We propose to estimate phase heights using only inexpensive volume flow measurements.
  arXiv  Detail & Related papers  (2026-01-26T11:57:28Z)
• A Semi-supervised Molecular Learning Framework for Activity Cliff Estimation [10.640733919289643]
  We propose a novel semi-supervised learning (SSL) method dubbed SemiMol.<n>SemiMol employs predictions on numerous unannotated data as pseudo-signals for subsequent training.<n>We show that SemiMol significantly enhances graph-based ML architectures and outpasses state-of-the-art pretraining and SSL baselines.
  arXiv  Detail & Related papers  (2026-01-08T02:20:25Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Toward Understanding BERT-Like Pre-Training for DNA Foundation Models [78.48760388079523]
  Existing pre-training methods for DNA sequences rely on direct adoptions of BERT pre-training from NLP. We introduce a novel approach called RandomMask, which gradually increases the task difficulty of BERT-like pre-training by continuously expanding its mask boundary. RandomMask achieves a staggering 68.16% in Matthew's correlation coefficient for Epigenetic Mark Prediction, a groundbreaking increase of 19.85% over the baseline.
  arXiv  Detail & Related papers  (2023-10-11T16:40:57Z)
• Unifying supervised learning and VAEs -- coverage, systematics and goodness-of-fit in normalizing-flow based neural network models for astro-particle reconstructions [0.0]
  Statistical uncertainties, coverage, systematic uncertainties or a goodness-of-fit measure are often not calculated. We show that a KL-divergence objective of the joint distribution of data and labels allows to unify supervised learning and variational autoencoders. We discuss how to calculate coverage probabilities without numerical integration for specific "base-ordered" contours.
  arXiv  Detail & Related papers  (2020-08-13T11:28:57Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.