Sparse Gaussian Processes with Spherical Harmonic Features

• URL: http://arxiv.org/abs/2006.16649v1
• Date: Tue, 30 Jun 2020 10:19:32 GMT
• Title: Sparse Gaussian Processes with Spherical Harmonic Features
• Authors: Vincent Dutordoir, Nicolas Durrande, James Hensman
• Abstract summary: We introduce a new class of inter-domain variational Gaussian processes (GP) Our inference scheme is comparable to variational Fourier features, but it does not suffer from the curse of dimensionality. Our experiments show that our model is able to fit a regression model for a dataset with 6 million entries two orders of magnitude faster.
• Score: 14.72311048788194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a new class of inter-domain variational Gaussian processes (GP) where data is mapped onto the unit hypersphere in order to use spherical harmonic representations. Our inference scheme is comparable to variational Fourier features, but it does not suffer from the curse of dimensionality, and leads to diagonal covariance matrices between inducing variables. This enables a speed-up in inference, because it bypasses the need to invert large covariance matrices. Our experiments show that our model is able to fit a regression model for a dataset with 6 million entries two orders of magnitude faster compared to standard sparse GPs, while retaining state of the art accuracy. We also demonstrate competitive performance on classification with non-conjugate likelihoods.

Related papers

• Learning Sparse High-Dimensional Matrix-Valued Graphical Models From Dependent Data [12.94486861344922]
  We consider the problem of inferring the conditional independence graph (CIG) of a sparse, high-dimensional, stationary matrix- Gaussian time series. We consider a sparse-based formulation of the problem with a Kronecker-decomposable power spectral density (PSD) We illustrate our approach using numerical examples utilizing both synthetic and real data.
  arXiv  Detail & Related papers  (2024-04-29T19:32:50Z)
• Solving High Frequency and Multi-Scale PDEs with Gaussian Processes [18.190228010565367]
  PINNs often struggle to solve high-frequency and multi-scale PDEs. We resort to the Gaussian process (GP) framework to solve this problem. We use Kronecker product properties and multilinear algebra to promote computational efficiency and scalability.
  arXiv  Detail & Related papers  (2023-11-08T05:26:58Z)
• Implicit Manifold Gaussian Process Regression [49.0787777751317]
  Gaussian process regression is widely used to provide well-calibrated uncertainty estimates. It struggles with high-dimensional data because of the implicit low-dimensional manifold upon which the data actually lies. In this paper we propose a technique capable of inferring implicit structure directly from data (labeled and unlabeled) in a fully differentiable way.
  arXiv  Detail & Related papers  (2023-10-30T09:52:48Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• Manifold Gaussian Variational Bayes on the Precision Matrix [70.44024861252554]
  We propose an optimization algorithm for Variational Inference (VI) in complex models. We develop an efficient algorithm for Gaussian Variational Inference whose updates satisfy the positive definite constraint on the variational covariance matrix. Due to its black-box nature, MGVBP stands as a ready-to-use solution for VI in complex models.
  arXiv  Detail & Related papers  (2022-10-26T10:12:31Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference [9.468270453795409]
  We study the doubly formulation of the BayesianVM model amenable with minibatch training. We show how this framework is compatible with different latent variable formulations and perform experiments to compare a suite of models. We demonstrate how we can train in the presence of massively missing data and obtain high-fidelity reconstructions.
  arXiv  Detail & Related papers  (2022-02-25T21:21:51Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• Laplace Matching for fast Approximate Inference in Generalized Linear Models [27.70274403550477]
  We propose an approximate inference framework primarily designed to be computationally cheap while still achieving high approximation quality. The concept, which we call emphLaplace Matching, involves closed-form, approximate, bi-directional transformations between the parameter spaces of exponential families. This effectively turns inference in GLMs into conjugate inference (with small approximation errors)
  arXiv  Detail & Related papers  (2021-05-07T08:25:17Z)
• tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder [0.0]
  tvGP-VAE is able to explicitly model correlation via the use of kernel functions. We show that the choice of which correlation structures to explicitly represent in the latent space has a significant impact on model performance.
  arXiv  Detail & Related papers  (2020-06-08T17:59:13Z)
• Gaussianization Flows [113.79542218282282]
  We propose a new type of normalizing flow model that enables both efficient iteration of likelihoods and efficient inversion for sample generation. Because of this guaranteed expressivity, they can capture multimodal target distributions without compromising the efficiency of sample generation.
  arXiv  Detail & Related papers  (2020-03-04T08:15:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.