An accuracy-runtime trade-off comparison of scalable Gaussian process approximations for spatial data

• URL: http://arxiv.org/abs/2501.11448v3
• Date: Wed, 30 Jul 2025 04:49:23 GMT
• Title: An accuracy-runtime trade-off comparison of scalable Gaussian process approximations for spatial data
• Authors: Filippo Rambelli, Fabio Sigrist,
• Abstract summary: We systematically compare the accuracy of different Gaussian process approximations concerning likelihood evaluation, parameter estimation, and prediction.<n>We find that Vecchia approximations consistently emerge as the most accurate in almost all experiments.
• Score: 11.141688859736805
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Gaussian processes (GPs) are flexible, probabilistic, non-parametric models widely employed in various fields such as spatial statistics and machine learning. A drawback of Gaussian processes is their computational cost having $\mathcal{O}(N^3)$ time and $\mathcal{O}(N^2)$ memory complexity which makes them prohibitive for large data sets. Numerous approximation techniques have been proposed to address this limitation. In this work, we systematically compare the accuracy of different Gaussian process approximations concerning likelihood evaluation, parameter estimation, and prediction taking into account the computational time required to perform these tasks. In other words, we analyze the trade-off between accuracy and runtime on multiple simulated and large-scale real-world data sets. We find that Vecchia approximations consistently emerge as the most accurate in almost all experiments.

Related papers

• Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference [55.150117654242706]
  We show that model selection for computation-aware GPs trained on 1.8 million data points can be done within a few hours on a single GPU. As a result of this work, Gaussian processes can be trained on large-scale datasets without significantly compromising their ability to quantify uncertainty.
  arXiv  Detail & Related papers  (2024-11-01T21:11:48Z)
• Linear cost and exponentially convergent approximation of Gaussian Matérn processes on intervals [43.341057405337295]
  computational cost for inference and prediction of statistical models based on Gaussian processes scales cubicly with the number of observations. We develop a method with linear cost and with a covariance error that decreases exponentially fast in the order $m$ of the proposed approximation. The method is based on an optimal rational approximation of the spectral density and results in an approximation that can be represented as a sum of $m$ independent Markov processes.
  arXiv  Detail & Related papers  (2024-10-16T19:57:15Z)
• Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data [9.913418444556486]
  We show how iterative methods can be used to reduce the computational costs for calculating likelihoods, gradients, and predictive distributions with FSAs.<n>We also present a novel, accurate, and fast way to calculate predictive variances relying on estimations and iterative methods.<n>All methods are implemented in a free C++ software library with high-level Python and R packages.
  arXiv  Detail & Related papers  (2024-05-23T12:25:22Z)
• Iterative Methods for Vecchia-Laplace Approximations for Latent Gaussian Process Models [11.141688859736805]
  We introduce and analyze several preconditioners, derive new convergence results, and propose novel methods for accurately approxing predictive variances.<n>In particular, we obtain a speed-up of an order of magnitude compared to Cholesky-based calculations.<n>All methods are implemented in a free C++ software library with high-level Python and R packages.
  arXiv  Detail & Related papers  (2023-10-18T14:31:16Z)
• Posterior Contraction Rates for Mat\'ern Gaussian Processes on Riemannian Manifolds [51.68005047958965]
  We show that intrinsic Gaussian processes can achieve better performance in practice. Our work shows that finer-grained analyses are needed to distinguish between different levels of data-efficiency.
  arXiv  Detail & Related papers  (2023-09-19T20:30:58Z)
• Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent [43.097493761380186]
  gradient algorithms are an efficient method of approximately solving linear systems. We show that gradient descent produces accurate predictions, even in cases where it does not converge quickly to the optimum. Experimentally, gradient descent achieves state-of-the-art performance on sufficiently large-scale or ill-conditioned regression tasks.
  arXiv  Detail & Related papers  (2023-06-20T15:07:37Z)
• Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models [69.22568644711113]
  We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversions. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.
  arXiv  Detail & Related papers  (2023-06-05T21:08:34Z)
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees [57.67528738886731]
  We study the numerical stability of scalable sparse approximations based on inducing points. For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
  arXiv  Detail & Related papers  (2022-10-14T15:20:17Z)
• Posterior and Computational Uncertainty in Gaussian Processes [52.26904059556759]
  Gaussian processes scale prohibitively with the size of the dataset. Many approximation methods have been developed, which inevitably introduce approximation error. This additional source of uncertainty, due to limited computation, is entirely ignored when using the approximate posterior. We develop a new class of methods that provides consistent estimation of the combined uncertainty arising from both the finite number of data observed and the finite amount of computation expended.
  arXiv  Detail & Related papers  (2022-05-30T22:16:25Z)
• MuyGPs: Scalable Gaussian Process Hyperparameter Estimation Using Local Cross-Validation [1.2233362977312945]
  We present MuyGPs, a novel efficient GP hyper parameter estimation method. MuyGPs builds upon prior methods that take advantage of the nearest neighbors structure of the data. We show that our method outperforms all known competitors both in terms of time-to-solution and the root mean squared error of the predictions.
  arXiv  Detail & Related papers  (2021-04-29T18:10:21Z)
• Fast covariance parameter estimation of spatial Gaussian process models using neural networks [0.0]
  We train NNs to take moderate size spatial fields or variograms as input and return the range and noise-to-signal covariance parameters. Once trained, the NNs provide estimates with a similar accuracy compared to ML estimation and at a speedup by a factor of 100 or more. This work can be easily extended to other, more complex, spatial problems and provides a proof-of-concept for this use of machine learning in computational statistics.
  arXiv  Detail & Related papers  (2020-12-30T22:06:26Z)
• AutoSimulate: (Quickly) Learning Synthetic Data Generation [70.82315853981838]
  We propose an efficient alternative for optimal synthetic data generation based on a novel differentiable approximation of the objective. We demonstrate that the proposed method finds the optimal data distribution faster (up to $50times$), with significantly reduced training data generation (up to $30times$) and better accuracy ($+8.7%$) on real-world test datasets than previous methods.
  arXiv  Detail & Related papers  (2020-08-16T11:36:11Z)
• Real-Time Regression with Dividing Local Gaussian Processes [62.01822866877782]
  Local Gaussian processes are a novel, computationally efficient modeling approach based on Gaussian process regression. Due to an iterative, data-driven division of the input space, they achieve a sublinear computational complexity in the total number of training points in practice. A numerical evaluation on real-world data sets shows their advantages over other state-of-the-art methods in terms of accuracy as well as prediction and update speed.
  arXiv  Detail & Related papers  (2020-06-16T18:43:31Z)
• Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties [12.068153197381575]
  We propose a novel variational family that allows for retaining covariances between latent processes while achieving fast convergence. We provide an efficient implementation of our new approach and apply it to several benchmark datasets. It yields excellent results and strikes a better balance between accuracy and calibrated uncertainty estimates than its state-of-the-art alternatives.
  arXiv  Detail & Related papers  (2020-05-22T11:10:59Z)
• Global Optimization of Gaussian processes [52.77024349608834]
  We propose a reduced-space formulation with trained Gaussian processes trained on few data points. The approach also leads to significantly smaller and computationally cheaper sub solver for lower bounding. In total, we reduce time convergence by orders of orders of the proposed method.
  arXiv  Detail & Related papers  (2020-05-21T20:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.