Scalable Gaussian Processes: Advances in Iterative Methods and Pathwise Conditioning

• URL: http://arxiv.org/abs/2507.06839v1
• Date: Wed, 09 Jul 2025 13:39:37 GMT
• Title: Scalable Gaussian Processes: Advances in Iterative Methods and Pathwise Conditioning
• Authors: Jihao Andreas Lin,
• Abstract summary: This dissertation focuses on the powerful combination of iterative methods and pathwise conditioning to develop methodological contributions.<n>By combining these two techniques, expensive computations are expressed as solutions to systems of linear equations and obtained by leveraging iterative linear system solvers.<n>This drastically reduces memory requirements, facilitating application to significantly larger amounts of data, and introduces matrix multiplication as the main computational operation.
• Score: 0.26107298043931204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Gaussian processes are a powerful framework for uncertainty-aware function approximation and sequential decision-making. Unfortunately, their classical formulation does not scale gracefully to large amounts of data and modern hardware for massively-parallel computation, prompting many researchers to develop techniques which improve their scalability. This dissertation focuses on the powerful combination of iterative methods and pathwise conditioning to develop methodological contributions which facilitate the use of Gaussian processes in modern large-scale settings. By combining these two techniques synergistically, expensive computations are expressed as solutions to systems of linear equations and obtained by leveraging iterative linear system solvers. This drastically reduces memory requirements, facilitating application to significantly larger amounts of data, and introduces matrix multiplication as the main computational operation, which is ideal for modern hardware.

Related papers

• A quantum gradient descent algorithm for optimizing Gaussian Process models [28.16587217223671]
  We propose a quantum gradient descent algorithm to optimize the Gaussian Process model.<n>Our algorithm achieves exponential speedup in computing the gradients of the log marginal likelihood.
  arXiv  Detail & Related papers  (2025-03-22T14:14:31Z)
• Linearly Convergent Mixup Learning [0.0]
  We present two novel algorithms that extend to a broader range of binary classification models.<n>Unlike gradient-based approaches, our algorithms do not require hyper parameters like learning rates, simplifying their implementation and optimization.<n>Our algorithms achieve faster convergence to the optimal solution compared to descent gradient approaches, and that mixup data augmentation consistently improves the predictive performance across various loss functions.
  arXiv  Detail & Related papers  (2025-01-14T02:33:40Z)
• Deep Learning-Enhanced Preconditioning for Efficient Conjugate Gradient Solvers in Large-Scale PDE Systems [11.712093849918123]
  This paper introduces a novel approach that integrates Graph Neural Network (GNN) with traditional IC.<n> Experimental results demonstrate an average reduction in iteration counts by 24.8% compared to IC.<n>The approach demon-strates robust generalization capabilities across scales.
  arXiv  Detail & Related papers  (2024-12-10T02:34:13Z)
• The Stochastic Conjugate Subgradient Algorithm For Kernel Support Vector Machines [1.738375118265695]
  This paper proposes an innovative method specifically designed for kernel support vector machines (SVMs) It not only achieves faster iteration per iteration but also exhibits enhanced convergence when compared to conventional SFO techniques. Our experimental results demonstrate that the proposed algorithm not only maintains but potentially exceeds the scalability of SFO methods.
  arXiv  Detail & Related papers  (2024-07-30T17:03:19Z)
• Machine Learning Insides OptVerse AI Solver: Design Principles and Applications [74.67495900436728]
  We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
  arXiv  Detail & Related papers  (2024-01-11T15:02:15Z)
• GloptiNets: Scalable Non-Convex Optimization with Certificates [61.50835040805378]
  We present a novel approach to non-cube optimization with certificates, which handles smooth functions on the hypercube or on the torus. By exploiting the regularity of the target function intrinsic in the decay of its spectrum, we allow at the same time to obtain precise certificates and leverage the advanced and powerful neural networks.
  arXiv  Detail & Related papers  (2023-06-26T09:42:59Z)
• Linear Self-Attention Approximation via Trainable Feedforward Kernel [77.34726150561087]
  In pursuit of faster computation, Efficient Transformers demonstrate an impressive variety of approaches. We aim to expand the idea of trainable kernel methods to approximate the self-attention mechanism of the Transformer architecture.
  arXiv  Detail & Related papers  (2022-11-08T08:14:11Z)
• AI-enhanced iterative solvers for accelerating the solution of large scale parametrized linear systems of equations [0.0]
  This paper exploits up-to-date ML tools and delivers customized iterative solvers of linear equation systems. The results indicate its superiority over conventional iterative solution schemes.
  arXiv  Detail & Related papers  (2022-07-06T09:47:14Z)
• Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces [96.53463532832939]
  We develop techniques for broadening the applicability of Gaussian processes. We introduce a wide class of efficient approximations built from this viewpoint. We develop a collection of Gaussian process models over non-Euclidean spaces.
  arXiv  Detail & Related papers  (2022-02-22T01:42:57Z)
• A Survey on Large-scale Machine Learning [67.6997613600942]
  Machine learning can provide deep insights into data, allowing machines to make high-quality predictions. Most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. Large-scale Machine Learning aims to learn patterns from big data with comparable performance efficiently.
  arXiv  Detail & Related papers  (2020-08-10T06:07:52Z)
• Global Optimization of Gaussian processes [52.77024349608834]
  We propose a reduced-space formulation with trained Gaussian processes trained on few data points. The approach also leads to significantly smaller and computationally cheaper sub solver for lower bounding. In total, we reduce time convergence by orders of orders of the proposed method.
  arXiv  Detail & Related papers  (2020-05-21T20:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.