On Representations of Mean-Field Variational Inference

• URL: http://arxiv.org/abs/2210.11385v1
• Date: Thu, 20 Oct 2022 16:26:22 GMT
• Title: On Representations of Mean-Field Variational Inference
• Authors: Soumyadip Ghosh and Yingdong Lu and Tomasz Nowicki and Edith Zhang
• Abstract summary: We present a framework to analyze mean field variational inference (MFVI) algorithms. Our approach enables the MFVI problem to be represented in three different manners. Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm yields a gradient flow in the limit.
• Score: 2.4316550366482357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The mean field variational inference (MFVI) formulation restricts the general Bayesian inference problem to the subspace of product measures. We present a framework to analyze MFVI algorithms, which is inspired by a similar development for general variational Bayesian formulations. Our approach enables the MFVI problem to be represented in three different manners: a gradient flow on Wasserstein space, a system of Fokker-Planck-like equations and a diffusion process. Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm in the product Wasserstein space of measures yields a gradient flow in the limit. A similar result is obtained for their associated densities, with the limit being given by a quasi-linear partial differential equation. A popular class of practical algorithms falls in this framework, which provides tools to establish convergence. We hope this framework could be used to guarantee convergence of algorithms in a variety of approaches, old and new, to solve variational inference problems.

Related papers

• A Variational Bayesian Inference Theory of Elasticity and Its Mixed Probabilistic Finite Element Method for Inverse Deformation Solutions in Any Dimension [3.9900555221077396]
  The elastic strain energy is used as a prior in a Bayesian inference network. The proposed method is able to inversely predict continuum deformation mappings with strong discontinuity or fracture.
  arXiv  Detail & Related papers  (2024-10-10T04:35:18Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• A Unified Theory of Stochastic Proximal Point Methods without Smoothness [52.30944052987393]
  Proximal point methods have attracted considerable interest owing to their numerical stability and robustness against imperfect tuning. This paper presents a comprehensive analysis of a broad range of variations of the proximal point method (SPPM)
  arXiv  Detail & Related papers  (2024-05-24T21:09:19Z)
• A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods [14.845158804951552]
  We establish the first mathematically rigorous link between Bayesian, variational Bayesian, and ensemble methods. On a technical level, our contribution amounts to a generalised variational inference through the lense of Wasserstein flows.
  arXiv  Detail & Related papers  (2023-05-24T11:13:59Z)
• SARAH-based Variance-reduced Algorithm for Stochastic Finite-sum Cocoercive Variational Inequalities [137.6408511310322]
  We consider the problem of finite-sum cocoercive variational inequalities. For strongly monotone problems it is possible to achieve linear convergence to a solution using this method.
  arXiv  Detail & Related papers  (2022-10-12T08:04:48Z)
• Error-Aware Spatial Ensembles for Video Frame Interpolation [50.63021118973639]
  Video frame(VFI) algorithms have improved considerably in recent years due to unprecedented progress in both data-driven algorithms and their implementations. Recent research has introduced advanced motion estimation or novel warping methods as the means to address challenging VFI scenarios. This work introduces such a solution. By closely examining the correlation between optical flow and IE, the paper proposes novel error prediction metrics that partition the middle frame into distinct regions corresponding to different IE levels.
  arXiv  Detail & Related papers  (2022-07-25T16:15:38Z)
• Mean-field Variational Inference via Wasserstein Gradient Flow [8.05603983337769]
  Variational inference, such as the mean-field (MF) approximation, requires certain conjugacy structures for efficient computation. We introduce a general computational framework to implement MFal inference for Bayesian models, with or without latent variables, using the Wasserstein gradient flow (WGF) We propose a new constraint-free function approximation method using neural networks to numerically realize our algorithm.
  arXiv  Detail & Related papers  (2022-07-17T04:05:32Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Low-Discrepancy Points via Energetic Variational Inference [5.936959130012709]
  We propose a deterministic variational inference approach and generate low-discrepancy points by minimizing the kernel discrepancy. We name the resulting algorithm EVI-MMD and demonstrate it through examples in which the target distribution is fully specified. Its performances are satisfactory compared to alternative methods in the applications of distribution approximation, numerical integration, and generative learning.
  arXiv  Detail & Related papers  (2021-11-21T03:09:07Z)
• Probabilistic Numerical Method of Lines for Time-Dependent Partial Differential Equations [20.86460521113266]
  Current state-of-the-art PDE solvers treat the space- and time-dimensions separately, serially, and with black-box algorithms. We introduce a probabilistic version of a technique called method of lines to fix this issue. Joint quantification of space- and time-uncertainty becomes possible without losing the performance benefits of well-tuned ODE solvers.
  arXiv  Detail & Related papers  (2021-10-22T15:26:05Z)
• Optimal oracle inequalities for solving projected fixed-point equations [53.31620399640334]
  We study methods that use a collection of random observations to compute approximate solutions by searching over a known low-dimensional subspace of the Hilbert space. We show how our results precisely characterize the error of a class of temporal difference learning methods for the policy evaluation problem with linear function approximation.
  arXiv  Detail & Related papers  (2020-12-09T20:19:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.