Maximizing Efficiency of Dataset Compression for Machine Learning Potentials With Information Theory

• URL: http://arxiv.org/abs/2511.10561v1
• Date: Fri, 14 Nov 2025 01:57:45 GMT
• Title: Maximizing Efficiency of Dataset Compression for Machine Learning Potentials With Information Theory
• Authors: Benjamin Yu, Vincenzo Lordi, Daniel Schwalbe-Koda,
• Abstract summary: Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations.<n>Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on.<n>We develop an information-theoretical framework to quantify the efficiency of dataset compression methods.
• Score: 0.45880283710344066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets, and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

Related papers

• SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models [51.74498855100541]
  Large language models (LLMs) have shown strong reasoning capabilities when fine-tuned with reinforcement learning (RL)<n>We propose textbfSPaRFT, a self-paced learning framework that enables efficient learning based on the capability of the model being trained.
  arXiv  Detail & Related papers  (2025-08-07T03:50:48Z)
• Efficient Dataset Distillation through Low-Rank Space Sampling [34.29086540681496]
  This paper proposes a dataset distillation method based on Matching Training Trajectories with Low-rank Space Sampling.<n>The synthetic data is represented by basis vectors and shared dimension mappers from these subspaces.<n>The proposed method is tested on CIFAR-10, CIFAR-100, and SVHN datasets, and outperforms the baseline methods by an average of 9.9%.
  arXiv  Detail & Related papers  (2025-03-11T02:59:17Z)
• Leveraging Large Language Models to Address Data Scarcity in Machine Learning: Applications in Graphene Synthesis [0.0]
  Machine learning in materials science faces challenges due to limited experimental data.<n>We propose strategies that utilize large language models (LLMs) to enhance machine learning performance.
  arXiv  Detail & Related papers  (2025-03-06T16:04:01Z)
• DUPRE: Data Utility Prediction for Efficient Data Valuation [49.60564885180563]
  Cooperative game theory-based data valuation, such as Data Shapley, requires evaluating the data utility and retraining the ML model for multiple data subsets.<n>Our framework, textttDUPRE, takes an alternative yet complementary approach that reduces the cost per subset evaluation by predicting data utilities instead of evaluating them by model retraining.<n>Specifically, given the evaluated data utilities of some data subsets, textttDUPRE fits a emphGaussian process (GP) regression model to predict the utility of every other data subset.
  arXiv  Detail & Related papers  (2025-02-22T08:53:39Z)
• Data-Efficient Machine Learning Potentials via Difference Vectors Based on Local Atomic Environments [13.307935336307475]
  We propose a new method--Difference Vectors based on Local Atomic Environments (DV-LAE)<n>DV-LAE encodes structural differences via histogram-based descriptors and enables visual analysis through t-S dimensionality reduction.<n>We demonstrate that DV-LAE significantly reduces dataset size and training time across various materials systems.
  arXiv  Detail & Related papers  (2025-01-27T04:08:37Z)
• Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information [43.44508080585033]
  We introduce conditional mutual information (CMI) to assess the class-aware complexity of a dataset.<n>We minimize the distillation loss while constraining the class-aware complexity of the synthetic dataset.
  arXiv  Detail & Related papers  (2024-12-13T08:10:47Z)
• Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth [83.15263499262824]
  We prove that gradient descent converges to a solution that completely disregards the sparse structure of the input. We show how to improve upon Gaussian performance for the compression of sparse data by adding a denoising function to a shallow architecture. We validate our findings on image datasets, such as CIFAR-10 and MNIST.
  arXiv  Detail & Related papers  (2024-02-07T16:32:29Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Improved Distribution Matching for Dataset Condensation [91.55972945798531]
  We propose a novel dataset condensation method based on distribution matching. Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
  arXiv  Detail & Related papers  (2023-07-19T04:07:33Z)
• The Bearable Lightness of Big Data: Towards Massive Public Datasets in Scientific Machine Learning [0.0]
  We show that lossy compression algorithms offer a realistic pathway for exposing high-fidelity scientific data to open-source data repositories. In this paper, we outline, construct, and evaluate the requirements for establishing a big data framework.
  arXiv  Detail & Related papers  (2022-07-25T21:44:53Z)
• Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
  We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV) NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
  arXiv  Detail & Related papers  (2022-06-23T09:35:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.