Projected Wasserstein gradient descent for high-dimensional Bayesian inference

• URL: http://arxiv.org/abs/2102.06350v2
• Date: Mon, 15 Feb 2021 02:16:10 GMT
• Title: Projected Wasserstein gradient descent for high-dimensional Bayesian inference
• Authors: Yifei Wang, Peng Chen and Wuchen Li
• Abstract summary: We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. We overcome this challenge by exploiting the intrinsic low-rank structure in the difference between the posterior and prior distributions.
• Score: 8.750791391857264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. The underlying density function of a particle system of WGD is approximated by kernel density estimation (KDE), which faces the long-standing curse of dimensionality. We overcome this challenge by exploiting the intrinsic low-rank structure in the difference between the posterior and prior distributions. The parameters are projected into a low-dimensional subspace to alleviate the approximation error of KDE in high dimensions. We formulate a projected Wasserstein gradient flow and analyze its convergence property under mild assumptions. Several numerical experiments illustrate the accuracy, convergence, and complexity scalability of pWGD with respect to parameter dimension, sample size, and processor cores.

Related papers

• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• Differentially Private Gradient Flow based on the Sliced Wasserstein Distance [59.1056830438845]
  We introduce a novel differentially private generative modeling approach based on a gradient flow in the space of probability measures. Experiments show that our proposed model can generate higher-fidelity data at a low privacy budget.
  arXiv  Detail & Related papers  (2023-12-13T15:47:30Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• Multilevel CNNs for Parametric PDEs [0.0]
  We combine concepts from multilevel solvers for partial differential equations with neural network based deep learning. An in-depth theoretical analysis shows that the proposed architecture is able to approximate multigrid V-cycles to arbitrary precision. We find substantial improvements over state-of-the-art deep learning-based solvers.
  arXiv  Detail & Related papers  (2023-04-01T21:11:05Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization [43.6961980403682]
  The computation of Wasserstein gradient direction is essential for posterior sampling problems and scientific computing. We study the variational problem in the family of two-layer networks with squared-ReLU activations, towards which we derive a semi-definite programming (SDP) relaxation. This SDP can be viewed as an approximation of the Wasserstein gradient in a broader function family including two-layer networks.
  arXiv  Detail & Related papers  (2022-05-26T00:51:12Z)
• Large-Scale Wasserstein Gradient Flows [84.73670288608025]
  We introduce a scalable scheme to approximate Wasserstein gradient flows. Our approach relies on input neural networks (ICNNs) to discretize the JKO steps. As a result, we can sample from the measure at each step of the gradient diffusion and compute its density.
  arXiv  Detail & Related papers  (2021-06-01T19:21:48Z)
• Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs [6.178864935410097]
  We construct surrogates for high-dimensional PDE-governed parametric maps in the form of projected neural networks. We demonstrate that the proposed projected neural network achieves greater accuracy than a full neural network.
  arXiv  Detail & Related papers  (2020-11-30T18:46:40Z)
• Augmented Sliced Wasserstein Distances [55.028065567756066]
  We propose a new family of distance metrics, called augmented sliced Wasserstein distances (ASWDs) ASWDs are constructed by first mapping samples to higher-dimensional hypersurfaces parameterized by neural networks. Numerical results demonstrate that the ASWD significantly outperforms other Wasserstein variants for both synthetic and real-world problems.
  arXiv  Detail & Related papers  (2020-06-15T23:00:08Z)
• High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces [31.663702435594825]
  Density estimation plays a key role in many tasks in machine learning, statistical inference, and visualization. Main bottleneck in high-dimensional density estimation is the prohibitive computational cost and the slow convergence rate. We propose novel estimators for high-dimensional non-parametric density estimation called the adaptive hyperbolic cross density estimators.
  arXiv  Detail & Related papers  (2020-06-05T21:27:59Z)
• Projected Stein Variational Gradient Descent [8.359602391273755]
  We propose a projected Stein variational descent gradient (pSVGD) method to overcome the challenge of intrinsic low dimensionality of the data informed subspace. We adaptively construct the subspace using a gradient information matrix of the log-likelihood, and apply pSVGD to the much lower-dimensional coefficients of the parameter projection.
  arXiv  Detail & Related papers  (2020-02-09T23:17:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.