Transfer Learning for Deep Learning-based Prediction of Lattice Thermal Conductivity

• URL: http://arxiv.org/abs/2411.18259v1
• Date: Wed, 27 Nov 2024 11:57:58 GMT
• Title: Transfer Learning for Deep Learning-based Prediction of Lattice Thermal Conductivity
• Authors: L. Klochko, M. d'Aquin, A. Togo, L. Chaput,
• Abstract summary: We study the impact of transfer learning on the precision and generalizability of a deep learning model (ParAIsite) We show that a much greater improvement is obtained when first fine-tuning it on a large datasets of low-quality approximations of lattice thermal conductivity (LTC) The promising results pave the way towards a greater ability to explore large databases in search of low thermal conductivity materials.
• Score: 0.0
• License:
• Abstract: Machine learning promises to accelerate the material discovery by enabling high-throughput prediction of desirable macro-properties from atomic-level descriptors or structures. However, the limited data available about precise values of these properties have been a barrier, leading to predictive models with limited precision or the ability to generalize. This is particularly true of lattice thermal conductivity (LTC): existing datasets of precise (ab initio, DFT-based) computed values are limited to a few dozen materials with little variability. Based on such datasets, we study the impact of transfer learning on both the precision and generalizability of a deep learning model (ParAIsite). We start from an existing model (MEGNet~\cite{Chen2019}) and show that improvements are obtained by fine-tuning a pre-trained version on different tasks. Interestingly, we also show that a much greater improvement is obtained when first fine-tuning it on a large datasets of low-quality approximations of LTC (based on the AGL model) and then applying a second phase of fine-tuning with our high-quality, smaller-scale datasets. The promising results obtained pave the way not only towards a greater ability to explore large databases in search of low thermal conductivity materials but also to methods enabling increasingly precise predictions in areas where quality data are rare.

Related papers

• Towards Data-Efficient Pretraining for Atomic Property Prediction [51.660835328611626]
  We show that pretraining on a task-relevant dataset can match or surpass large-scale pretraining. We introduce the Chemical Similarity Index (CSI), a novel metric inspired by computer vision's Fr'echet Inception Distance.
  arXiv  Detail & Related papers  (2025-02-16T11:46:23Z)
• An Investigation on Machine Learning Predictive Accuracy Improvement and Uncertainty Reduction using VAE-based Data Augmentation [2.517043342442487]
  Deep generative learning uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data.
  arXiv  Detail & Related papers  (2024-10-24T18:15:48Z)
• Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties [10.955525128731654]
  Large language models (LLMs) bear promise as a fast and accurate material modeling paradigm for evaluation, analysis, and design. We present a physics-based training pipeline that tackles the pathology of data scarcity.
  arXiv  Detail & Related papers  (2024-07-03T02:57:40Z)
• Low-rank finetuning for LLMs: A fairness perspective [54.13240282850982]
  Low-rank approximation techniques have become the de facto standard for fine-tuning Large Language Models. This paper investigates the effectiveness of these methods in capturing the shift of fine-tuning datasets from the initial pre-trained data distribution. We show that low-rank fine-tuning inadvertently preserves undesirable biases and toxic behaviors.
  arXiv  Detail & Related papers  (2024-05-28T20:43:53Z)
• Transfer Learning for Molecular Property Predictions from Small Data Sets [0.0]
  We benchmark common machine learning models for the prediction of molecular properties on two small data sets. We present a transfer learning strategy that uses large data sets to pre-train the respective models and allows to obtain more accurate models after fine-tuning on the original data sets.
  arXiv  Detail & Related papers  (2024-04-20T14:25:34Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• End-to-end Material Thermal Conductivity Prediction through Machine Learning [1.5565958456748663]
  Machine learning models for thermal conductivity prediction suffer from overfitting. Best mean absolute percentage error achieved on the test dataset remained in the range of 50-60%.
  arXiv  Detail & Related papers  (2023-11-06T14:34:30Z)
• Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line [65.14099135546594]
  Recent test-time adaptation (TTA) methods drastically strengthen the ACL and AGL trends in models, even in shifts where models showed very weak correlations before. Our results show that by combining TTA with AGL-based estimation methods, we can estimate the OOD performance of models with high precision for a broader set of distribution shifts.
  arXiv  Detail & Related papers  (2023-10-07T23:21:25Z)
• To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis [50.31589712761807]
  Large language models (LLMs) are notoriously token-hungry during pre-training, and high-quality text data on the web is approaching its scaling limit for LLMs. We investigate the consequences of repeating pre-training data, revealing that the model is susceptible to overfitting. Second, we examine the key factors contributing to multi-epoch degradation, finding that significant factors include dataset size, model parameters, and training objectives.
  arXiv  Detail & Related papers  (2023-05-22T17:02:15Z)
• Multi-fidelity prediction of fluid flow and temperature field based on transfer learning using Fourier Neural Operator [10.104417481736833]
  This work proposes a novel multi-fidelity learning method based on the Fourier Neural Operator. It uses abundant low-fidelity data and limited high-fidelity data under transfer learning paradigm. Three typical fluid and temperature prediction problems are chosen to validate the accuracy of the proposed multi-fidelity model.
  arXiv  Detail & Related papers  (2023-04-14T07:46:03Z)
• Churn Reduction via Distillation [54.5952282395487]
  We show an equivalence between training with distillation using the base model as the teacher and training with an explicit constraint on the predictive churn. We then show that distillation performs strongly for low churn training against a number of recent baselines.
  arXiv  Detail & Related papers  (2021-06-04T18:03:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.