Active Learning for Deep Gaussian Process Surrogates
- URL: http://arxiv.org/abs/2012.08015v1
- Date: Tue, 15 Dec 2020 00:09:37 GMT
- Title: Active Learning for Deep Gaussian Process Surrogates
- Authors: Annie Sauer, Robert B. Gramacy, David Higdon
- Abstract summary: Deep Gaussian processes (DGPs) are increasingly popular as predictive models in machine learning (ML)
Here we explore DGPs as surrogates for computer simulation experiments whose response surfaces exhibit similar characteristics.
We build up the design sequentially, limiting both expensive evaluation of the simulator code and mitigating cubic costs of DGP inference.
- Score: 0.3222802562733786
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Gaussian processes (DGPs) are increasingly popular as predictive models
in machine learning (ML) for their non-stationary flexibility and ability to
cope with abrupt regime changes in training data. Here we explore DGPs as
surrogates for computer simulation experiments whose response surfaces exhibit
similar characteristics. In particular, we transport a DGP's automatic warping
of the input space and full uncertainty quantification (UQ), via a novel
elliptical slice sampling (ESS) Bayesian posterior inferential scheme, through
to active learning (AL) strategies that distribute runs non-uniformly in the
input space -- something an ordinary (stationary) GP could not do. Building up
the design sequentially in this way allows smaller training sets, limiting both
expensive evaluation of the simulator code and mitigating cubic costs of DGP
inference. When training data sizes are kept small through careful acquisition,
and with parsimonious layout of latent layers, the framework can be both
effective and computationally tractable. Our methods are illustrated on
simulation data and two real computer experiments of varying input
dimensionality. We provide an open source implementation in the "deepgp"
package on CRAN.
Related papers
- The Power of Resets in Online Reinforcement Learning [73.64852266145387]
We explore the power of simulators through online reinforcement learning with local simulator access (or, local planning)
We show that MDPs with low coverability can be learned in a sample-efficient fashion with only $Qstar$-realizability.
We show that the notorious Exogenous Block MDP problem is tractable under local simulator access.
arXiv Detail & Related papers (2024-04-23T18:09:53Z) - Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts.
We propose a test-time Forward-Optimization Adaptation (FOA) method.
FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
arXiv Detail & Related papers (2024-04-02T05:34:33Z) - Revisiting Active Sets for Gaussian Process Decoders [0.0]
We develop a new estimate of the log-marginal likelihood based on recently discovered links to cross-validation.
We demonstrate that the resulting active sets (SAS) approximation significantly improves the robustness of GP decoder training.
arXiv Detail & Related papers (2022-09-10T10:49:31Z) - Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them.
NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
arXiv Detail & Related papers (2021-10-26T10:45:25Z) - Modular Gaussian Processes for Transfer Learning [0.0]
We present a framework for transfer learning based on modular variational Gaussian processes (GP)
We develop a module-based method that builds ensemble GP models without revisiting any data.
Our method avoids undesired data centralisation, reduces rising computational costs and allows the transfer of learned uncertainty metrics after training.
arXiv Detail & Related papers (2021-10-26T09:15:18Z) - Incremental Ensemble Gaussian Processes [53.3291389385672]
We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary.
With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions.
The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
arXiv Detail & Related papers (2021-10-13T15:11:25Z) - Subset-of-Data Variational Inference for Deep Gaussian-Processes
Regression [0.0]
Deep Gaussian Processes (DGPs) are multi-layer, flexible extensions of Gaussian processes.
Sparse approximations simplify the training but often require optimization over a large number of inducing inputs and their locations.
In this paper, we simplify the training by setting the locations to a fixed subset of data and sampling the inducing inputs from a variational distribution.
arXiv Detail & Related papers (2021-07-17T15:55:35Z) - Deep Gaussian Process Emulation using Stochastic Imputation [0.0]
We propose a novel deep Gaussian process (DGP) inference method for computer model emulation using imputation.
Byally imputing the latent layers, the approach transforms the DGP into the linked GP, a state-of-the-art surrogate model formed by linking a system of feed-forward coupled GPs.
arXiv Detail & Related papers (2021-07-04T10:46:23Z) - Likelihood-Free Inference with Deep Gaussian Processes [70.74203794847344]
Surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations.
We propose a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions.
Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions and maintain a comparable performance in unimodal cases.
arXiv Detail & Related papers (2020-06-18T14:24:05Z) - Real-Time Regression with Dividing Local Gaussian Processes [62.01822866877782]
Local Gaussian processes are a novel, computationally efficient modeling approach based on Gaussian process regression.
Due to an iterative, data-driven division of the input space, they achieve a sublinear computational complexity in the total number of training points in practice.
A numerical evaluation on real-world data sets shows their advantages over other state-of-the-art methods in terms of accuracy as well as prediction and update speed.
arXiv Detail & Related papers (2020-06-16T18:43:31Z) - Scaled Vecchia approximation for fast computer-model emulation [0.6445605125467573]
We adapt and extend a powerful class of GP methods from spatial statistics to enable the scalable analysis and emulation of large computer experiments.
Our methods are highly scalable, enabling estimation, joint prediction and simulation in near-linear time in the number of model runs.
arXiv Detail & Related papers (2020-05-01T14:08:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.