Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators

• URL: http://arxiv.org/abs/2410.12690v2
• Date: Thu, 17 Oct 2024 01:53:56 GMT
• Title: Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators
• Authors: Xinming Wang, Simon Mak, John Miller, Jianguo Wu,
• Abstract summary: A critical bottleneck for scientific progress is the costly nature of computer simulations for complex systems. In many applications, one often has available data on related systems. A key question is how information from such "source" systems can be transferred for effective surrogate training.
• Score: 8.507372217592472
• License:
• Abstract: A critical bottleneck for scientific progress is the costly nature of computer simulations for complex systems. Surrogate models provide an appealing solution: such models are trained on simulator evaluations, then used to emulate and quantify uncertainty on the expensive simulator at unexplored inputs. In many applications, one often has available data on related systems. For example, in designing a new jet turbine, there may be existing studies on turbines with similar configurations. A key question is how information from such "source" systems can be transferred for effective surrogate training on the "target" system of interest. We thus propose a new LOcal transfer Learning Gaussian Process (LOL-GP) model, which leverages a carefully-designed Gaussian process to transfer such information for surrogate modeling. The key novelty of the LOL-GP is a latent regularization model, which identifies regions where transfer should be performed and regions where it should be avoided. This "local transfer" property is desirable in scientific systems: at certain parameters, such systems may behave similarly and thus transfer is beneficial; at other parameters, they may behave differently and thus transfer is detrimental. By accounting for local transfer, the LOL-GP can rectify a critical limitation of "negative transfer" in existing transfer learning models, where the transfer of information worsens predictive performance. We derive a Gibbs sampling algorithm for efficient posterior predictive sampling on the LOL-GP, for both the multi-source and multi-fidelity transfer settings. We then show, via a suite of numerical experiments and an application for jet turbine design, the improved surrogate performance of the LOL-GP over existing methods.

Related papers

• Machine learning surrogates for efficient hydrologic modeling: Insights from stochastic simulations of managed aquifer recharge [0.0]
  We propose a hybrid modeling workflow for process-based hydrologic models and machine learning surrogate models. As a case study, we apply this workflow to simulations of variably saturated groundwater flow at a prospective managed aquifer recharge site. Our results demonstrate that ML surrogate models can achieve under 10% mean absolute percentage error and yield order-of-magnitude runtime savings.
  arXiv  Detail & Related papers  (2024-07-30T15:24:27Z)
• Transfer Learning Under High-Dimensional Graph Convolutional Regression Model for Node Classification [20.18595334666282]
  We propose a Graph Convolutional Multinomial Logistic Regression (GCR) model and a transfer learning method based on the GCR model, called Trans-GCR. We provide theoretical guarantees of the estimate obtained under GCR model in high-dimensional settings.
  arXiv  Detail & Related papers  (2024-05-26T19:30:14Z)
• Enhancing Polynomial Chaos Expansion Based Surrogate Modeling using a Novel Probabilistic Transfer Learning Strategy [2.980666177064344]
  In black-box simulations, non-intrusive PCE allows the construction of surrogates using a set of simulation response evaluations. We propose to leverage transfer learning whereby knowledge gained through similar PCE surrogate construction tasks is transferred to a new surrogate-construction task.
  arXiv  Detail & Related papers  (2023-12-07T19:16:42Z)
• Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop [1.8218298349840023]
  Reinforcement Learning has great potential for autonomously training agents to perform complex control tasks. To use RL effectively in embedded system function development, the generated agents must be able to handle real-world applications. This work focuses on accelerating the training process of RL agents by combining Transfer Learning (TL) and X-in-the-Loop (XiL) simulation.
  arXiv  Detail & Related papers  (2023-10-25T09:13:12Z)
• Emergent Agentic Transformer from Chain of Hindsight Experience [96.56164427726203]
  We show that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches. This is the first time that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
  arXiv  Detail & Related papers  (2023-05-26T00:43:02Z)
• Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design [68.1682448368636]
  We present a supervised pretraining approach to learn circuit representations that can be adapted to new unseen topologies or unseen prediction tasks. To cope with the variable topological structure of different circuits we describe each circuit as a graph and use graph neural networks (GNNs) to learn node embeddings. We show that pretraining GNNs on prediction of output node voltages can encourage learning representations that can be adapted to new unseen topologies or prediction of new circuit level properties.
  arXiv  Detail & Related papers  (2022-03-29T21:18:47Z)
• A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations [69.73803123972297]
  We propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems.
  arXiv  Detail & Related papers  (2020-12-21T17:39:51Z)
• Active Learning for Deep Gaussian Process Surrogates [0.3222802562733786]
  Deep Gaussian processes (DGPs) are increasingly popular as predictive models in machine learning (ML) Here we explore DGPs as surrogates for computer simulation experiments whose response surfaces exhibit similar characteristics. We build up the design sequentially, limiting both expensive evaluation of the simulator code and mitigating cubic costs of DGP inference.
  arXiv  Detail & Related papers  (2020-12-15T00:09:37Z)
• Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources [78.72922528736011]
  We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses. BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
  arXiv  Detail & Related papers  (2020-07-17T01:52:34Z)
• Likelihood-Free Inference with Deep Gaussian Processes [70.74203794847344]
  Surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations. We propose a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions. Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions and maintain a comparable performance in unimodal cases.
  arXiv  Detail & Related papers  (2020-06-18T14:24:05Z)
• RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real [74.45688231140689]
  We introduce the RL-scene consistency loss for image translation, which ensures that the translation operation is invariant with respect to the Q-values associated with the image. We obtain RL-CycleGAN, a new approach for simulation-to-real-world transfer for reinforcement learning.
  arXiv  Detail & Related papers  (2020-06-16T08:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.