Gaussian Process-based learning with new MCMC-based implementation of Wishart prior on correlation matrix

• URL: http://arxiv.org/abs/2605.27093v1
• Date: Tue, 26 May 2026 14:37:18 GMT
• Title: Gaussian Process-based learning with new MCMC-based implementation of Wishart prior on correlation matrix
• Abstract summary: We develop a self-assembled'' Wishart prior for the covariance matrix.<n>We also undertake Bayesian inference on the kernel hyper parameters using MCMC.<n>Results suggest that direct prior specification on the covariance matrix can be useful for diagnosing weakly informative inputs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In probabilstic supervised learning of an input-output relationship - as a sample function of a Gaussian Process (GP) - priors are typically specified for the hyperparameters of the kernel that parametrises the covariance function of the GP, where the induced covariance matrix of the (resulting multivariate Normal) likelihood, governs the learning and prediction. When the sought function is highly multivariate, multiple lengthscale parameters must be learnt simultaneously, making inference difficult. We develop a ``self-assembled'' Wishart prior for the covariance matrix, while undertaking Bayesian inference on the kernel hyperparameters using MCMC. The construction uses a look-back window over recent MCMC iterations to define a time-step dependent scale matrix, thereby introducing adaptiveness to the chain. Results suggest that direct prior specification on the covariance matrix can be useful for diagnosing weakly informative inputs within the GP-based learning paradigm. We support our prior development with two distinct empirical illustrations - one on synthetic data, and another on a real-world dataset.

Related papers

• Gaussian-Mixture-Model Q-Functions for Policy Iteration in Reinforcement Learning [7.056697401102689]
  This paper introduces a novel function-approximation role for Gaussian mixture models (GMMs) as direct surrogates for Q-function losses.<n>These parametric models, termed GMM-QFs, possess substantial representational capacity.<n>They are shown to be universal approximators over a broad class of functions.
  arXiv  Detail & Related papers  (2025-12-21T15:00:32Z)
• Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors [14.453654853392619]
  We establish in-expectation and tail bounds on the generalization error of representation learning type algorithms.<n>We propose a systematic approach to simultaneously learning a data-dependent Gaussian mixture prior and using it as a regularizer.
  arXiv  Detail & Related papers  (2025-02-21T15:43:31Z)
• A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning [74.80956524812714]
  We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning. These problems are often formalized as Bi-Level optimizations (BLO) We introduce a novel perspective by turning a given BLO problem into a ii optimization, where the inner loss function becomes a smooth distribution, and the outer loss becomes an expected loss over the inner distribution.
  arXiv  Detail & Related papers  (2024-10-14T12:10:06Z)
• Non-stationary and Sparsely-correlated Multi-output Gaussian Process with Spike-and-Slab Prior [0.0]
  Multi-output Gaussian process (MGP) is commonly used as a transfer learning method. This study proposes a non-stationary MGP model that can capture both the dynamic and sparse correlation among outputs.
  arXiv  Detail & Related papers  (2024-09-05T00:56:25Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• Manifold Gaussian Variational Bayes on the Precision Matrix [70.44024861252554]
  We propose an optimization algorithm for Variational Inference (VI) in complex models. We develop an efficient algorithm for Gaussian Variational Inference whose updates satisfy the positive definite constraint on the variational covariance matrix. Due to its black-box nature, MGVBP stands as a ready-to-use solution for VI in complex models.
  arXiv  Detail & Related papers  (2022-10-26T10:12:31Z)
• On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach [0.76146285961466]
  Gaussian processes (GPs) are frequently used in machine learning and statistics to construct powerful models. We propose a pseudo-marginal (PM) scheme that offers exact inference as well as computational gains through doubly estimators for the likelihood and large datasets.
  arXiv  Detail & Related papers  (2021-03-04T20:48:29Z)
• Gaussian Process Latent Class Choice Models [7.992550355579791]
  We present a non-parametric class of probabilistic machine learning within discrete choice models (DCMs) The proposed model would assign individuals probabilistically to behaviorally homogeneous clusters (latent classes) using GPs. The model is tested on two different mode choice applications and compared against different LCCM benchmarks.
  arXiv  Detail & Related papers  (2021-01-28T19:56:42Z)
• Kernel learning approaches for summarising and combining posterior similarity matrices [68.8204255655161]
  We build upon the notion of the posterior similarity matrix (PSM) in order to suggest new approaches for summarising the output of MCMC algorithms for Bayesian clustering models. A key contribution of our work is the observation that PSMs are positive semi-definite, and hence can be used to define probabilistically-motivated kernel matrices.
  arXiv  Detail & Related papers  (2020-09-27T14:16:14Z)
• Understanding Implicit Regularization in Over-Parameterized Single Index Model [55.41685740015095]
  We design regularization-free algorithms for the high-dimensional single index model. We provide theoretical guarantees for the induced implicit regularization phenomenon.
  arXiv  Detail & Related papers  (2020-07-16T13:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.