Algorithms and Scientific Software for Quasi-Monte Carlo, Fast Gaussian Process Regression, and Scientific Machine Learning

• URL: http://arxiv.org/abs/2511.21915v1
• Date: Wed, 26 Nov 2025 21:11:08 GMT
• Title: Algorithms and Scientific Software for Quasi-Monte Carlo, Fast Gaussian Process Regression, and Scientific Machine Learning
• Authors: Aleksei G. Sorokin,
• Abstract summary: This thesis unifies our developments in three broad domains: Quasi-Monte Carlo (QMC) methods for efficient high-dimensional integration, Gaussian process (GP) regression for high-dimensional domains with built-in uncertainty, and scientific machine learning (sciML) for modeling partial differential equations (PDEs) with mesh-free solvers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most scientific domains elicit the development of efficient algorithms and accessible scientific software. This thesis unifies our developments in three broad domains: Quasi-Monte Carlo (QMC) methods for efficient high-dimensional integration, Gaussian process (GP) regression for high-dimensional interpolation with built-in uncertainty quantification, and scientific machine learning (sciML) for modeling partial differential equations (PDEs) with mesh-free solvers. For QMC, we built new algorithms for vectorized error estimation and developed QMCPy (https://qmcsoftware.github.io/QMCSoftware/): an open-source Python interface to randomized low-discrepancy sequence generators, automatic variable transforms, adaptive error estimation procedures, and diverse use cases. For GPs, we derived new digitally-shift-invariant kernels of higher-order smoothness, developed novel fast multitask GP algorithms, and produced the scalable Python software FastGPs (https://alegresor.github.io/fastgps/). For sciML, we developed a new algorithm capable of machine precision recovery of PDEs with random coefficients. We have also studied a number of applications including GPs for probability of failure estimation, multilevel GPs for the Darcy flow equation, neural surrogates for modeling radiative transfer, and fast GPs for Bayesian multilevel QMC.

Related papers

• STRIDE: Sparse Techniques for Regression in Deep Gaussian Processes [0.3277163122167433]
  We develop a particle-based expectation expectation training method for deep GP training on large-scale data.<n>We test our method on standard benchmark problems.
  arXiv  Detail & Related papers  (2025-05-16T15:18:15Z)
• HiGP: A high-performance Python package for Gaussian Process [13.127443064937735]
  HiGP is a high-performance Python package designed for efficient Gaussian Process regression (GPR) and classification (GPC)<n>It implements various effective matrix-vector (MatVec) and matrix-matrix (MatMul) multiplication strategies specifically tailored for kernel matrices.<n>With a user-friendly Python interface, HiGP integrates seamlessly with PyTorch and other Python packages, allowing easy incorporation into existing machine learning and data analysis.
  arXiv  Detail & Related papers  (2025-03-04T04:17:36Z)
• Optimal Kernel Learning for Gaussian Process Models with High-Dimensional Input [0.0]
  In some simulation models, the outputs may only be significantly influenced by a small subset of the input variables, referred to as the active variables''<n>We propose an optimal kernel learning approach to identify these active variables, thereby overcoming GP model limitations and enhancing system understanding.
  arXiv  Detail & Related papers  (2025-02-23T15:39:59Z)
• Gradients of Functions of Large Matrices [18.361820028457718]
  We show how to differentiate workhorses of numerical linear algebra efficiently. We derive previously unknown adjoint systems for Lanczos and Arnoldi iterations, implement them in JAX, and show that the resulting code can compete with Diffrax. All this is achieved without any problem-specific code optimisation.
  arXiv  Detail & Related papers  (2024-05-27T15:39:45Z)
• Algorithms for perturbative analysis and simulation of quantum dynamics [0.0]
  We develop general purpose algorithms for computing and utilizing both the Dyson series and Magnus expansion. We demonstrate how to use these tools to approximate fidelity in a region of model parameter space. We show how the pre-computation step can be phrased as a multivariable expansion problem with fewer terms than in the original method.
  arXiv  Detail & Related papers  (2022-10-20T21:07:47Z)
• A hybrid data driven-physics constrained Gaussian process regression framework with deep kernel for uncertainty quantification [21.972192114861873]
  We propose a hybrid data driven-physics constrained Gaussian process regression framework. We encode the physics knowledge with Boltzmann-Gibbs distribution and derive our model through maximum likelihood (ML) approach. The proposed model achieves good results in high-dimensional problem, and correctly propagate the uncertainty, with very limited labelled data provided.
  arXiv  Detail & Related papers  (2022-05-13T07:53:49Z)
• Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization [61.26619639722804]
  We propose a conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms. The proposed method, equipped with an average gradient (SAG) estimator, requires only one sample per iteration. Nevertheless, it guarantees fast convergence rates on par with more sophisticated variance reduction techniques.
  arXiv  Detail & Related papers  (2022-02-26T19:10:48Z)
• Scaling Gaussian Process Optimization by Evaluating a Few Unique Candidates Multiple Times [119.41129787351092]
  We show that sequential black-box optimization based on GPs can be made efficient by sticking to a candidate solution for multiple evaluation steps. We modify two well-established GP-Opt algorithms, GP-UCB and GP-EI to adapt rules from batched GP-Opt.
  arXiv  Detail & Related papers  (2022-01-30T20:42:14Z)
• Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
  We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
  arXiv  Detail & Related papers  (2021-10-26T10:45:25Z)
• Predictive Coding Approximates Backprop along Arbitrary Computation Graphs [68.8204255655161]
  We develop a strategy to translate core machine learning architectures into their predictive coding equivalents. Our models perform equivalently to backprop on challenging machine learning benchmarks. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry.
  arXiv  Detail & Related papers  (2020-06-07T15:35:47Z)
• Global Optimization of Gaussian processes [52.77024349608834]
  We propose a reduced-space formulation with trained Gaussian processes trained on few data points. The approach also leads to significantly smaller and computationally cheaper sub solver for lower bounding. In total, we reduce time convergence by orders of orders of the proposed method.
  arXiv  Detail & Related papers  (2020-05-21T20:59:11Z)
• Learning Gaussian Graphical Models via Multiplicative Weights [54.252053139374205]
  We adapt an algorithm of Klivans and Meka based on the method of multiplicative weight updates. The algorithm enjoys a sample complexity bound that is qualitatively similar to others in the literature. It has a low runtime $O(mp2)$ in the case of $m$ samples and $p$ nodes, and can trivially be implemented in an online manner.
  arXiv  Detail & Related papers  (2020-02-20T10:50:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.