Efficiently Sampling Functions from Gaussian Process Posteriors
- URL: http://arxiv.org/abs/2002.09309v4
- Date: Sun, 16 Aug 2020 13:37:40 GMT
- Title: Efficiently Sampling Functions from Gaussian Process Posteriors
- Authors: James T. Wilson and Viacheslav Borovitskiy and Alexander Terenin and
Peter Mostowsky and Marc Peter Deisenroth
- Abstract summary: We propose an easy-to-use and general-purpose approach for fast posterior sampling.
We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
- Score: 76.94808614373609
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Gaussian processes are the gold standard for many real-world modeling
problems, especially in cases where a model's success hinges upon its ability
to faithfully represent predictive uncertainty. These problems typically exist
as parts of larger frameworks, wherein quantities of interest are ultimately
defined by integrating over posterior distributions. These quantities are
frequently intractable, motivating the use of Monte Carlo methods. Despite
substantial progress in scaling up Gaussian processes to large training sets,
methods for accurately generating draws from their posterior distributions
still scale cubically in the number of test locations. We identify a
decomposition of Gaussian processes that naturally lends itself to scalable
sampling by separating out the prior from the data. Building off of this
factorization, we propose an easy-to-use and general-purpose approach for fast
posterior sampling, which seamlessly pairs with sparse approximations to afford
scalability both during training and at test time. In a series of experiments
designed to test competing sampling schemes' statistical properties and
practical ramifications, we demonstrate how decoupled sample paths accurately
represent Gaussian process posteriors at a fraction of the usual cost.
Related papers
- von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
We explore a family of expressive and interpretable distributions over circle-valued random functions.
The resulting probability model has connections with continuous spin models in statistical physics.
For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
arXiv Detail & Related papers (2024-06-19T01:57:21Z) - A Flow-Based Generative Model for Rare-Event Simulation [0.483420384410068]
We present a method in which a Normalizing Flow generative model is trained to simulate samples directly from a conditional distribution.
We illustrate that by simulating directly from a rare-event distribution significant insight can be gained into the way rare events happen.
arXiv Detail & Related papers (2023-05-13T08:25:57Z) - Mixtures of Gaussian process experts based on kernel stick-breaking
processes [0.6396288020763143]
We propose a new mixture model of Gaussian process experts based on kernel stick-breaking processes.
Our model maintains the intuitive appeal yet improve the performance of the existing models.
The model behaviour and improved predictive performance are demonstrated in experiments using six datasets.
arXiv Detail & Related papers (2023-04-26T21:23:01Z) - Robust Gaussian Process Regression with Huber Likelihood [2.7184224088243365]
We propose a robust process model in the Gaussian process framework with the likelihood of observed data expressed as the Huber probability distribution.
The proposed model employs weights based on projection statistics to scale residuals and bound the influence of vertical outliers and bad leverage points on the latent functions estimates.
arXiv Detail & Related papers (2023-01-19T02:59:33Z) - Convergence for score-based generative modeling with polynomial
complexity [9.953088581242845]
We prove the first convergence guarantees for the core mechanic behind Score-based generative modeling.
Compared to previous works, we do not incur error that grows exponentially in time or that suffers from a curse of dimensionality.
We show that a predictor-corrector gives better convergence than using either portion alone.
arXiv Detail & Related papers (2022-06-13T14:57:35Z) - Gaussian Processes and Statistical Decision-making in Non-Euclidean
Spaces [96.53463532832939]
We develop techniques for broadening the applicability of Gaussian processes.
We introduce a wide class of efficient approximations built from this viewpoint.
We develop a collection of Gaussian process models over non-Euclidean spaces.
arXiv Detail & Related papers (2022-02-22T01:42:57Z) - Sensing Cox Processes via Posterior Sampling and Positive Bases [56.82162768921196]
We study adaptive sensing of point processes, a widely used model from spatial statistics.
We model the intensity function as a sample from a truncated Gaussian process, represented in a specially constructed positive basis.
Our adaptive sensing algorithms use Langevin dynamics and are based on posterior sampling (textscCox-Thompson) and top-two posterior sampling (textscTop2) principles.
arXiv Detail & Related papers (2021-10-21T14:47:06Z) - Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
We take a two-step approach by first modeling the probability distribution and then sampling from that model.
We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
arXiv Detail & Related papers (2021-10-20T12:25:22Z) - Pathwise Conditioning of Gaussian Processes [72.61885354624604]
Conventional approaches for simulating Gaussian process posteriors view samples as draws from marginal distributions of process values at finite sets of input locations.
This distribution-centric characterization leads to generative strategies that scale cubically in the size of the desired random vector.
We show how this pathwise interpretation of conditioning gives rise to a general family of approximations that lend themselves to efficiently sampling Gaussian process posteriors.
arXiv Detail & Related papers (2020-11-08T17:09:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.