Improving neural network predictions of material properties with limited data using transfer learning

• URL: http://arxiv.org/abs/2006.16420v1
• Date: Mon, 29 Jun 2020 22:34:30 GMT
• Title: Improving neural network predictions of material properties with limited data using transfer learning
• Authors: Schuyler Krawczuk and Daniele Venturi
• Abstract summary: We develop new transfer learning algorithms to accelerate prediction of material properties from ab initio simulations. Transfer learning has been successfully utilized for data-efficient modeling in applications other than materials science.
• Score: 3.2851683371946754
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop new transfer learning algorithms to accelerate prediction of material properties from ab initio simulations based on density functional theory (DFT). Transfer learning has been successfully utilized for data-efficient modeling in applications other than materials science, and it allows transferable representations learned from large datasets to be repurposed for learning new tasks even with small datasets. In the context of materials science, this opens the possibility to develop generalizable neural network models that can be repurposed on other materials, without the need of generating a large (computationally expensive) training set of materials properties. The proposed transfer learning algorithms are demonstrated on predicting the Gibbs free energy of light transition metal oxides.

Related papers

• Scaling Law of Sim2Real Transfer Learning in Expanding Computational Materials Databases for Real-World Predictions [13.20562263181952]
  Fine-tuning a predictor pretrained on a computational database to a real system can result in models with outstanding generalization capabilities. This study demonstrates the scaling law of simulation-to-real (Sim2Real) transfer learning for several machine learning tasks in materials science.
  arXiv  Detail & Related papers  (2024-08-07T18:47:58Z)
• Advancing Extrapolative Predictions of Material Properties through Learning to Learn [1.3274508420845539]
  We use attention-based architecture of neural networks and meta-learning algorithms to acquire extrapolative generalization capability. We highlight the potential of such extrapolatively trained models, particularly with their ability to rapidly adapt to unseen material domains.
  arXiv  Detail & Related papers  (2024-03-25T09:30:19Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Mobile Traffic Prediction at the Edge through Distributed and Transfer Learning [2.687861184973893]
  The research in this topic concentrated on making predictions in a centralized fashion, by collecting data from the different network elements. We propose a novel prediction framework based on edge computing which uses datasets obtained on the edge through a large measurement campaign.
  arXiv  Detail & Related papers  (2023-10-22T23:48:13Z)
• Joint Feature and Differentiable $ k $-NN Graph Learning using Dirichlet Energy [103.74640329539389]
  We propose a deep FS method that simultaneously conducts feature selection and differentiable $ k $-NN graph learning. We employ Optimal Transport theory to address the non-differentiability issue of learning $ k $-NN graphs in neural networks. We validate the effectiveness of our model with extensive experiments on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-05-21T08:15:55Z)
• Advancing Reacting Flow Simulations with Data-Driven Models [50.9598607067535]
  Key to effective use of machine learning tools in multi-physics problems is to couple them to physical and computer models. The present chapter reviews some of the open opportunities for the application of data-driven reduced-order modeling of combustion systems.
  arXiv  Detail & Related papers  (2022-09-05T16:48:34Z)
• Transfer Learning with Deep Tabular Models [66.67017691983182]
  We show that upstream data gives tabular neural networks a decisive advantage over GBDT models. We propose a realistic medical diagnosis benchmark for tabular transfer learning. We propose a pseudo-feature method for cases where the upstream and downstream feature sets differ.
  arXiv  Detail & Related papers  (2022-06-30T14:24:32Z)
• A transfer learning enhanced the physics-informed neural network model for vortex-induced vibration [0.0]
  This paper proposed a transfer learning enhanced the physics-informed neural network (PINN) model to study the VIV (2D) The physics-informed neural network, when used in conjunction with the transfer learning method, enhances learning efficiency and keeps predictability in the target task by common characteristics knowledge from the source model without requiring a huge quantity of datasets.
  arXiv  Detail & Related papers  (2021-12-29T08:20:23Z)
• Audacity of huge: overcoming challenges of data scarcity and data quality for machine learning in computational materials discovery [1.0036312061637764]
  Machine learning (ML)-accelerated discovery requires large amounts of high-fidelity data to reveal predictive structure-property relationships. For many properties of interest in materials discovery, the challenging nature and high cost of data generation has resulted in a data landscape that is scarcely populated and of dubious quality. In the absence of manual curation, increasingly sophisticated natural language processing and automated image analysis are making it possible to learn structure-property relationships from the literature.
  arXiv  Detail & Related papers  (2021-11-02T21:43:58Z)
• TRAIL: Near-Optimal Imitation Learning with Suboptimal Data [100.83688818427915]
  We present training objectives that use offline datasets to learn a factored transition model. Our theoretical analysis shows that the learned latent action space can boost the sample-efficiency of downstream imitation learning. To learn the latent action space in practice, we propose TRAIL (Transition-Reparametrized Actions for Imitation Learning), an algorithm that learns an energy-based transition model.
  arXiv  Detail & Related papers  (2021-10-27T21:05:00Z)
• Multilinear Compressive Learning with Prior Knowledge [106.12874293597754]
  Multilinear Compressive Learning (MCL) framework combines Multilinear Compressive Sensing and Machine Learning into an end-to-end system. Key idea behind MCL is the assumption of the existence of a tensor subspace which can capture the essential features from the signal for the downstream learning task. In this paper, we propose a novel solution to address both of the aforementioned requirements, i.e., How to find those tensor subspaces in which the signals of interest are highly separable?
  arXiv  Detail & Related papers  (2020-02-17T19:06:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.