Variational Inference as Iterative Projection in a Bayesian Hilbert Space with Application to Robotic State Estimation

• URL: http://arxiv.org/abs/2005.07275v3
• Date: Mon, 26 Sep 2022 16:33:05 GMT
• Title: Variational Inference as Iterative Projection in a Bayesian Hilbert Space with Application to Robotic State Estimation
• Authors: Timothy D. Barfoot and Gabriele M. T. D'Eleuterio
• Abstract summary: Variational Bayesian inference is an important machine-learning tool that finds application from statistics to robotics. We show that variational inference based on KL divergence can amount to iterative projection, in the Euclidean sense, of the Bayesian posterior onto a subspace corresponding to the selected approximation family.
• Score: 14.670851095242451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Variational Bayesian inference is an important machine-learning tool that finds application from statistics to robotics. The goal is to find an approximate probability density function (PDF) from a chosen family that is in some sense 'closest' to the full Bayesian posterior. Closeness is typically defined through the selection of an appropriate loss functional such as the Kullback-Leibler (KL) divergence. In this paper, we explore a new formulation of variational inference by exploiting the fact that (most) PDFs are members of a Bayesian Hilbert space under careful definitions of vector addition, scalar multiplication and an inner product. We show that, under the right conditions, variational inference based on KL divergence can amount to iterative projection, in the Euclidean sense, of the Bayesian posterior onto a subspace corresponding to the selected approximation family. We work through the details of this general framework for the specific case of the Gaussian approximation family and show the equivalence to another Gaussian variational inference approach. We furthermore discuss the implications for systems that exhibit sparsity, which is handled naturally in Bayesian space, and give an example of a high-dimensional robotic state estimation problem that can be handled as a result. We provide some preliminary examples of how the approach could be applied to non-Gaussian inference and discuss the limitations of the approach in detail to encourage follow-on work along these lines.

Related papers

• von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
  We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Analytical Approximation of the ELBO Gradient in the Context of the Clutter Problem [0.0]
  We propose an analytical solution for approximating the gradient of the Evidence Lower Bound (ELBO) in variational inference problems. The proposed method demonstrates good accuracy and rate of convergence together with linear computational complexity.
  arXiv  Detail & Related papers  (2024-04-16T13:19:46Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Robust probabilistic inference via a constrained transport metric [8.85031165304586]
  We offer a novel alternative by constructing an exponentially tilted empirical likelihood carefully designed to concentrate near a parametric family of distributions. The proposed approach finds applications in a wide variety of robust inference problems, where we intend to perform inference on the parameters associated with the centering distribution. We demonstrate superior performance of our methodology when compared against state-of-the-art robust Bayesian inference methods.
  arXiv  Detail & Related papers  (2023-03-17T16:10:06Z)
• Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning [9.106412307976067]
  We develop a framework for generalized variational inference in infinite-dimensional function spaces. We use it to construct a method termed Gaussian Wasserstein inference (GWI) An exciting application of GWI is the ability to use deep neural networks in the variational parametrisation of GWI.
  arXiv  Detail & Related papers  (2022-05-12T20:10:31Z)
• Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence [77.06203118175335]
  Variational Inference (VI) is a popular alternative to exact sampling in Bayesian inference. Importance sampling (IS) is often used to fine-tune and de-bias the estimates of approximate Bayesian inference procedures. We propose a novel combination of optimization and sampling techniques for approximate Bayesian inference.
  arXiv  Detail & Related papers  (2021-06-30T11:00:24Z)
• Variational inference with a quantum computer [0.0]
  Inference is the task of drawing conclusions about unobserved variables given observations of related variables. One alternative is variational inference, where a candidate probability distribution is optimized to approximate the posterior distribution over unobserved variables. In this work, we propose quantum Born machines as variational distributions over discrete variables.
  arXiv  Detail & Related papers  (2021-03-11T15:12:21Z)
• Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
  Inferring the parameters of a model based on experimental observations is central to the scientific method. A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations. We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
  arXiv  Detail & Related papers  (2021-02-12T12:23:13Z)
• Optimal oracle inequalities for solving projected fixed-point equations [53.31620399640334]
  We study methods that use a collection of random observations to compute approximate solutions by searching over a known low-dimensional subspace of the Hilbert space. We show how our results precisely characterize the error of a class of temporal difference learning methods for the policy evaluation problem with linear function approximation.
  arXiv  Detail & Related papers  (2020-12-09T20:19:32Z)
• Understanding Variational Inference in Function-Space [20.940162027560408]
  We highlight some advantages and limitations of employing the Kullback-Leibler divergence in this setting. We propose (featurized) Bayesian linear regression as a benchmark for function-space' inference methods that directly measures approximation quality.
  arXiv  Detail & Related papers  (2020-11-18T17:42:01Z)
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization [56.69671152009899]
  We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization. We also propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction.
  arXiv  Detail & Related papers  (2020-02-20T15:13:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.