Posterior Sampling with Denoising Oracles via Tilted Transport

• URL: http://arxiv.org/abs/2407.00745v1
• Date: Sun, 30 Jun 2024 16:11:42 GMT
• Title: Posterior Sampling with Denoising Oracles via Tilted Transport
• Authors: Joan Bruna, Jiequn Han,
• Abstract summary: We introduce the textittilted transport technique, which leverages the quadratic structure of the log-likelihood in linear inverse problems. We quantify the conditions under which this boosted posterior is strongly log-concave, highlighting the dependencies on the condition number of the measurement matrix. The resulting posterior sampling scheme is shown to reach the computational threshold predicted for sampling Ising models.
• Score: 37.14320147233444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Score-based diffusion models have significantly advanced high-dimensional data generation across various domains, by learning a denoising oracle (or score) from datasets. From a Bayesian perspective, they offer a realistic modeling of data priors and facilitate solving inverse problems through posterior sampling. Although many heuristic methods have been developed recently for this purpose, they lack the quantitative guarantees needed in many scientific applications. In this work, we introduce the \textit{tilted transport} technique, which leverages the quadratic structure of the log-likelihood in linear inverse problems in combination with the prior denoising oracle to transform the original posterior sampling problem into a new `boosted' posterior that is provably easier to sample from. We quantify the conditions under which this boosted posterior is strongly log-concave, highlighting the dependencies on the condition number of the measurement matrix and the signal-to-noise ratio. The resulting posterior sampling scheme is shown to reach the computational threshold predicted for sampling Ising models [Kunisky'23] with a direct analysis, and is further validated on high-dimensional Gaussian mixture models and scalar field $\varphi^4$ models.

Related papers

• Posterior sampling via Langevin dynamics based on generative priors [31.84543941736757]
  Posterior sampling in high-dimensional spaces using generative models holds significant promise for various applications. Existing methods require restarting the entire generative process for each new sample, making the procedure computationally expensive. We propose efficient posterior sampling by simulating Langevin dynamics in the noise space of a pre-trained generative model.
  arXiv  Detail & Related papers  (2024-10-02T22:57:47Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models [98.95988351420334]
  We present the first framework to solve linear inverse problems leveraging pre-trained latent diffusion models. We theoretically analyze our algorithm showing provable sample recovery in a linear model setting.
  arXiv  Detail & Related papers  (2023-07-02T17:21:30Z)
• Refining Amortized Posterior Approximations using Gradient-Based Summary Statistics [0.9176056742068814]
  We present an iterative framework to improve the amortized approximations of posterior distributions in the context of inverse problems. We validate our method in a controlled setting by applying it to a stylized problem, and observe improved posterior approximations with each iteration.
  arXiv  Detail & Related papers  (2023-05-15T15:47:19Z)
• Score-based Diffusion Models in Function Space [140.792362459734]
  Diffusion models have recently emerged as a powerful framework for generative modeling. We introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Posterior samples of source galaxies in strong gravitational lenses with score-based priors [107.52670032376555]
  We use a score-based model to encode the prior for the inference of undistorted images of background galaxies. We show how the balance between the likelihood and the prior meet our expectations in an experiment with out-of-distribution data.
  arXiv  Detail & Related papers  (2022-11-07T19:00:42Z)
• From Denoising Diffusions to Denoising Markov Models [38.33676858989955]
  Denoising diffusions are state-of-the-art generative models exhibiting remarkable empirical performance. We propose a unifying framework generalising this approach to a wide class of spaces and leading to an original extension of score matching.
  arXiv  Detail & Related papers  (2022-11-07T14:34:27Z)
• Diffusion Posterior Sampling for General Noisy Inverse Problems [50.873313752797124]
  We extend diffusion solvers to handle noisy (non)linear inverse problems via approximation of the posterior sampling. Our method demonstrates that diffusion models can incorporate various measurement noise statistics.
  arXiv  Detail & Related papers  (2022-09-29T11:12:27Z)
• Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods [35.24886589614034]
  We consider quadratic definite Ising models on the hypercube with a general interaction $J$. Our general result implies the first time sampling algorithms for low-rank Ising models.
  arXiv  Detail & Related papers  (2022-02-17T21:43:50Z)
• Efficiently Sampling Functions from Gaussian Process Posteriors [76.94808614373609]
  We propose an easy-to-use and general-purpose approach for fast posterior sampling. We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
  arXiv  Detail & Related papers  (2020-02-21T14:03:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.