Conditioning of Banach Space Valued Gaussian Random Variables: An Approximation Approach Based on Martingales

• URL: http://arxiv.org/abs/2404.03453v3
• Date: Tue, 6 Aug 2024 07:53:09 GMT
• Title: Conditioning of Banach Space Valued Gaussian Random Variables: An Approximation Approach Based on Martingales
• Authors: Ingo Steinwart,
• Abstract summary: We investigate the conditional distributions of two Banach space valued, jointly Gaussian random variables. We show that their means and covariances are determined by a general finite dimensional approximation scheme based upon a martingale approach.
• Score: 8.81121308982678
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we investigate the conditional distributions of two Banach space valued, jointly Gaussian random variables. We show that these conditional distributions are again Gaussian and that their means and covariances are determined by a general finite dimensional approximation scheme based upon a martingale approach. In particular, it turns out that the covariance operators occurring in this scheme converge with respect to the nuclear norm and that the conditional probabilities converge weakly. Moreover, we discuss in detail, how our approximation scheme can be implemented in several classes of important Banach spaces such as (reproducing kernel) Hilbert spaces and spaces of continuous functions. As an example, we then apply our general results to the case of Gaussian processes with continuous paths conditioned to partial but infinite observations of their paths. Here we show that conditioning on sufficiently rich, increasing sets of finitely many observations leads to consistent approximations, that is, both the mean and covariance functions converge uniformly and the conditional probabilities converge weakly. Moreover, we discuss how these results improve our understanding of the popular Gaussian processes for machine learning.

Related papers

• Approximation of optimization problems with constraints through kernel Sum-Of-Squares [77.27820145069515]
  We show that pointwise inequalities are turned into equalities within a class of nonnegative kSoS functions. We also show that focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints.
  arXiv  Detail & Related papers  (2023-01-16T10:30:04Z)
• Kullback-Leibler and Renyi divergences in reproducing kernel Hilbert space and Gaussian process settings [0.0]
  We present formulations for regularized Kullback-Leibler and R'enyi divergences via the Alpha Log-Determinant (Log-Det) divergences. For characteristic kernels, the first setting leads to divergences between arbitrary Borel probability measures on a complete, separable metric space. We show that the Alpha Log-Det divergences are continuous in the Hilbert-Schmidt norm, which enables us to apply laws of large numbers for Hilbert space-valued random variables.
  arXiv  Detail & Related papers  (2022-07-18T06:40:46Z)
• Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces [96.53463532832939]
  We develop techniques for broadening the applicability of Gaussian processes. We introduce a wide class of efficient approximations built from this viewpoint. We develop a collection of Gaussian process models over non-Euclidean spaces.
  arXiv  Detail & Related papers  (2022-02-22T01:42:57Z)
• Information Theory with Kernel Methods [0.0]
  We show that the von Neumann entropy and relative entropy of these operators are intimately related to the usual notions of Shannon entropy and relative entropy. They come together with efficient estimation algorithms from various oracles on the probability distributions.
  arXiv  Detail & Related papers  (2022-02-17T09:42:29Z)
• Nearest neighbor empirical processes [7.034466417392574]
  An empirical measure based on the responses from the nearest neighbors to a given point $x$ is introduced and studied as a central statistical quantity. A uniform non-asymptotic bound is established under a well-known condition, often referred to as Vapnik-Chervonenkis, on the uniform entropy numbers. This suggests the possibility of using standard formulas to estimate the variance by using only the nearest neighbors instead of the full data.
  arXiv  Detail & Related papers  (2021-10-27T08:15:20Z)
• Estimation of Riemannian distances between covariance operators and Gaussian processes [0.7360807642941712]
  We study two distances between infinite-dimensional positive definite Hilbert-Schmidt operators. Results show that both distances converge in the Hilbert-Schmidt norm.
  arXiv  Detail & Related papers  (2021-08-26T09:57:47Z)
• The Connection between Discrete- and Continuous-Time Descriptions of Gaussian Continuous Processes [60.35125735474386]
  We show that discretizations yielding consistent estimators have the property of invariance under coarse-graining' This result explains why combining differencing schemes for derivatives reconstruction and local-in-time inference approaches does not work for time series analysis of second or higher order differential equations.
  arXiv  Detail & Related papers  (2021-01-16T17:11:02Z)
• Optimal oracle inequalities for solving projected fixed-point equations [53.31620399640334]
  We study methods that use a collection of random observations to compute approximate solutions by searching over a known low-dimensional subspace of the Hilbert space. We show how our results precisely characterize the error of a class of temporal difference learning methods for the policy evaluation problem with linear function approximation.
  arXiv  Detail & Related papers  (2020-12-09T20:19:32Z)
• Convergence of Gradient Algorithms for Nonconvex $C^{1+\alpha}$ Cost Functions [2.538209532048867]
  We show a byproduct that a gradient converges and provide an explicit upper bound on the convergence rate. This allows us to prove the almost sure convergence by Doobmartin's supergale convergence theorem.
  arXiv  Detail & Related papers  (2020-12-01T16:48:59Z)
• Pathwise Conditioning of Gaussian Processes [72.61885354624604]
  Conventional approaches for simulating Gaussian process posteriors view samples as draws from marginal distributions of process values at finite sets of input locations. This distribution-centric characterization leads to generative strategies that scale cubically in the size of the desired random vector. We show how this pathwise interpretation of conditioning gives rise to a general family of approximations that lend themselves to efficiently sampling Gaussian process posteriors.
  arXiv  Detail & Related papers  (2020-11-08T17:09:37Z)
• The Convergence Indicator: Improved and completely characterized parameter bounds for actual convergence of Particle Swarm Optimization [68.8204255655161]
  We introduce a new convergence indicator that can be used to calculate whether the particles will finally converge to a single point or diverge. Using this convergence indicator we provide the actual bounds completely characterizing parameter regions that lead to a converging swarm.
  arXiv  Detail & Related papers  (2020-06-06T19:08:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.