Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations

• URL: http://arxiv.org/abs/2408.06069v1
• Date: Mon, 12 Aug 2024 11:41:07 GMT
• Title: Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations
• Authors: Jian Xu, Zhiqi Lin, Min Chen, Junmei Yang, Delu Zeng, John Paisley,
• Abstract summary: We propose a fully Bayesian approach that treats the kernel hyper parameters as random variables and constructs coupled differential equations (SDEs) to learn their posterior distribution and that of inducing points. Our approach provides a time-varying, comprehensive, and realistic posterior approximation through coupling variables using SDE methods. Our work opens up exciting research avenues for advancing Bayesian inference and offers a powerful modeling tool for continuous-time Gaussian processes.
• Score: 7.439555720106548
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traditional deep Gaussian processes model the data evolution using a discrete hierarchy, whereas differential Gaussian processes (DIFFGPs) represent the evolution as an infinitely deep Gaussian process. However, prior DIFFGP methods often overlook the uncertainty of kernel hyperparameters and assume them to be fixed and time-invariant, failing to leverage the unique synergy between continuous-time models and approximate inference. In this work, we propose a fully Bayesian approach that treats the kernel hyperparameters as random variables and constructs coupled stochastic differential equations (SDEs) to learn their posterior distribution and that of inducing points. By incorporating estimation uncertainty on hyperparameters, our method enhances the model's flexibility and adaptability to complex dynamics. Additionally, our approach provides a time-varying, comprehensive, and realistic posterior approximation through coupling variables using SDE methods. Experimental results demonstrate the advantages of our method over traditional approaches, showcasing its superior performance in terms of flexibility, accuracy, and other metrics. Our work opens up exciting research avenues for advancing Bayesian inference and offers a powerful modeling tool for continuous-time Gaussian processes.

Related papers

• Variational Inference for SDEs Driven by Fractional Noise [16.434973057669676]
  We present a novel variational framework for performing inference in (neural) differential equations (SDEs) driven by Markov-approximate fractional Brownian motion (fBM) We propose the use of neural networks to learn the drift, diffusion and control terms within our variational posterior leading to the variational training of neural-SDEs.
  arXiv  Detail & Related papers  (2023-10-19T17:59:21Z)
• Multi-Response Heteroscedastic Gaussian Process Models and Their Inference [1.52292571922932]
  We propose a novel framework for the modeling of heteroscedastic covariance functions. We employ variational inference to approximate the posterior and facilitate posterior predictive modeling. We show that our proposed framework offers a robust and versatile tool for a wide array of applications.
  arXiv  Detail & Related papers  (2023-08-29T15:06:47Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• Variational Gaussian Process Diffusion Processes [17.716059928867345]
  Diffusion processes are a class of differential equations (SDEs) providing a rich family of expressive models. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach.
  arXiv  Detail & Related papers  (2023-06-03T09:43:59Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Counting Phases and Faces Using Bayesian Thermodynamic Integration [77.34726150561087]
  We introduce a new approach to reconstruction of the thermodynamic functions and phase boundaries in two-parametric statistical mechanics systems. We use the proposed approach to accurately reconstruct the partition functions and phase diagrams of the Ising model and the exactly solvable non-equilibrium TASEP.
  arXiv  Detail & Related papers  (2022-05-18T17:11:23Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Variational Inference for Continuous-Time Switching Dynamical Systems [29.984955043675157]
  We present a model based on an Markov jump process modulating a subordinated diffusion process. We develop a new continuous-time variational inference algorithm. We extensively evaluate our algorithm under the model assumption and for real-world examples.
  arXiv  Detail & Related papers  (2021-09-29T15:19:51Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features [0.0]
  We present a novel, domain-agnostic approach to tackling this problem. We use compositions of physics-informed random features, derived from ordinary differential equations. We find that our approach achieves comparable performance to a number of other probabilistic models on benchmark regression tasks.
  arXiv  Detail & Related papers  (2021-06-10T17:55:13Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.