Diffusion Bridge Variational Inference for Deep Gaussian Processes

• URL: http://arxiv.org/abs/2509.19078v2
• Date: Wed, 01 Oct 2025 09:29:45 GMT
• Title: Diffusion Bridge Variational Inference for Deep Gaussian Processes
• Authors: Jian Xu, Qibin Zhao, John Paisley, Delu Zeng,
• Abstract summary: Diffusion Bridge Variational Inference (DBVI) is a principled extension of Denoising diffusion variational inference (DDVI)<n>DBVI initiates the reverse diffusion from a learnable, data-dependent initial distribution.<n>It consistently outperforms DDVI and other variational baselines in predictive accuracy, convergence speed, and posterior quality.
• Score: 31.082191748525137
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Gaussian processes (DGPs) enable expressive hierarchical Bayesian modeling but pose substantial challenges for posterior inference, especially over inducing variables. Denoising diffusion variational inference (DDVI) addresses this by modeling the posterior as a time-reversed diffusion from a simple Gaussian prior. However, DDVI's fixed unconditional starting distribution remains far from the complex true posterior, resulting in inefficient inference trajectories and slow convergence. In this work, we propose Diffusion Bridge Variational Inference (DBVI), a principled extension of DDVI that initiates the reverse diffusion from a learnable, data-dependent initial distribution. This initialization is parameterized via an amortized neural network and progressively adapted using gradients from the ELBO objective, reducing the posterior gap and improving sample efficiency. To enable scalable amortization, we design the network to operate on the inducing inputs, which serve as structured, low-dimensional summaries of the dataset and naturally align with the inducing variables' shape. DBVI retains the mathematical elegance of DDVI, including Girsanov-based ELBOs and reverse-time SDEs,while reinterpreting the prior via a Doob-bridged diffusion process. We derive a tractable training objective under this formulation and implement DBVI for scalable inference in large-scale DGPs. Across regression, classification, and image reconstruction tasks, DBVI consistently outperforms DDVI and other variational baselines in predictive accuracy, convergence speed, and posterior quality.

Related papers

• DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing [5.215481191227242]
  We introduce textbfDAPS++, which allows the likelihood term to guide inference more directly while maintaining numerical stability.<n>textbfDAPS++ achieves high computational efficiency and robust reconstruction performance across diverse image restoration tasks.
  arXiv  Detail & Related papers  (2025-11-21T08:28:36Z)
• Diffusion State-Guided Projected Gradient for Inverse Problems [82.24625224110099]
  We propose Diffusion State-Guided Projected Gradient (DiffStateGrad) for inverse problems.<n>DiffStateGrad projects the measurement gradient onto a subspace that is a low-rank approximation of an intermediate state of the diffusion process.<n>We highlight that DiffStateGrad improves the robustness of diffusion models in terms of the choice of measurement guidance step size and noise.
  arXiv  Detail & Related papers  (2024-10-04T14:26:54Z)
• Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling [22.256068524699472]
  In this work, we propose an Annealed Importance Sampling (AIS) approach to address these issues. We combine the strengths of Sequential Monte Carlo samplers and VI to explore a wider range of posterior distributions and gradually approach the target distribution. Experimental results on both toy and image datasets demonstrate that our method outperforms state-of-the-art methods in terms of tighter variational bounds, higher log-likelihoods, and more robust convergence.
  arXiv  Detail & Related papers  (2024-08-13T08:09:05Z)
• Amortized Posterior Sampling with Diffusion Prior Distillation [55.03585818289934]
  Amortized Posterior Sampling is a novel variational inference approach for efficient posterior sampling in inverse problems.<n>Our method trains a conditional flow model to minimize the divergence between the variational distribution and the posterior distribution implicitly defined by the diffusion model.<n>Unlike existing methods, our approach is unsupervised, requires no paired training data, and is applicable to both Euclidean and non-Euclidean domains.
  arXiv  Detail & Related papers  (2024-07-25T09:53:12Z)
• Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems [12.482127049881026]
  We propose a novel approach to solve inverse problems with a diffusion prior from an amortized variational inference perspective. Our amortized inference learns a function that directly maps measurements to the implicit posterior distributions of corresponding clean data, enabling a single-step posterior sampling even for unseen measurements.
  arXiv  Detail & Related papers  (2024-07-23T02:14:18Z)
• Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors [11.01598521921903]
  DDVI is a black-box variational inference algorithm for latent variable models.<n>We introduce an expressive class of diffusion-based variational posteriors.<n>We train these posteriors with a novel regularized evidence lower bound.
  arXiv  Detail & Related papers  (2024-01-05T10:27:44Z)
• Distributed Variational Inference for Online Supervised Learning [15.038649101409804]
  This paper develops a scalable distributed probabilistic inference algorithm. It applies to continuous variables, intractable posteriors and large-scale real-time data in sensor networks.
  arXiv  Detail & Related papers  (2023-09-05T22:33:02Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems [64.29491112653905]
  We propose a novel and efficient diffusion sampling strategy that synergistically combines the diffusion sampling and Krylov subspace methods. Specifically, we prove that if tangent space at a denoised sample by Tweedie's formula forms a Krylov subspace, then the CG with the denoised data ensures the data consistency update to remain in the tangent space. Our proposed method achieves more than 80 times faster inference time than the previous state-of-the-art method.
  arXiv  Detail & Related papers  (2023-03-10T07:42:49Z)
• Variational Laplace Autoencoders [53.08170674326728]
  Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. We present a novel approach that addresses the limited posterior expressiveness of fully-factorized Gaussian assumption. We also present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models.
  arXiv  Detail & Related papers  (2022-11-30T18:59:27Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.