Reducing the cost of posterior sampling in linear inverse problems via task-dependent score learning

• URL: http://arxiv.org/abs/2405.15643v1
• Date: Fri, 24 May 2024 15:33:27 GMT
• Title: Reducing the cost of posterior sampling in linear inverse problems via task-dependent score learning
• Authors: Fabian Schneider, Duc-Lam Duong, Matti Lassas, Maarten V. de Hoop, Tapio Helin,
• Abstract summary: We show that the evaluation of the forward mapping can be entirely bypassed during posterior sample generation. We prove that this observation generalizes to the framework of infinite-dimensional diffusion models introduced recently.
• Score: 5.340736751238338
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Score-based diffusion models (SDMs) offer a flexible approach to sample from the posterior distribution in a variety of Bayesian inverse problems. In the literature, the prior score is utilized to sample from the posterior by different methods that require multiple evaluations of the forward mapping in order to generate a single posterior sample. These methods are often designed with the objective of enabling the direct use of the unconditional prior score and, therefore, task-independent training. In this paper, we focus on linear inverse problems, when evaluation of the forward mapping is computationally expensive and frequent posterior sampling is required for new measurement data, such as in medical imaging. We demonstrate that the evaluation of the forward mapping can be entirely bypassed during posterior sample generation. Instead, without introducing any error, the computational effort can be shifted to an offline task of training the score of a specific diffusion-like random process. In particular, the training is task-dependent requiring information about the forward mapping but not about the measurement data. It is shown that the conditional score corresponding to the posterior can be obtained from the auxiliary score by suitable affine transformations. We prove that this observation generalizes to the framework of infinite-dimensional diffusion models introduced recently and provide numerical analysis of the method. Moreover, we validate our findings with numerical experiments.

Related papers

• Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements [45.70011319850862]
  Diffusion models have emerged as a powerful foundation model for visual generation. Current posterior sampling based methods take the measurement into the posterior sampling to infer the distribution of the target data. We show that high-frequency information can be prematurely introduced during the early stages, which could induce larger posterior estimate errors. We propose a novel diffusion posterior sampling method DPS-CM, which incorporates a Crafted Measurement.
  arXiv  Detail & Related papers  (2024-11-15T00:06:57Z)
• Variational Diffusion Posterior Sampling with Midpoint Guidance [19.43399234028389]
  State-of-the-art approaches formulate the problem as that of sampling from a surrogate diffusion model targeting the posterior. While the former is replaced by the pre-trained score of the considered diffusion model, the guidance term has to be estimated. We validate the proposed approach through extensive experiments on linear and nonlinear inverse problems.
  arXiv  Detail & Related papers  (2024-10-13T18:03:53Z)
• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [52.98590762456236]
  Training-free test-time dynamic adapter (TDA) is a promising approach to address this issue. We propose a simple yet effective method for DistributiOnal Test-time Adaptation (Dota) Dota continually estimates the distributions of test samples, allowing the model to continually adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Tackling the Problem of Distributional Shifts: Correcting Misspecified, High-Dimensional Data-Driven Priors for Inverse Problems [39.58317527488534]
  Data-driven population-level distributions are emerging as an appealing alternative to simple parametric priors in inverse problems. It is difficult to acquire independent and identically distributed samples from the underlying data-generating process of interest to train these models. We show that starting from a misspecified prior distribution, the updated distribution becomes progressively closer to the underlying population-level distribution.
  arXiv  Detail & Related papers  (2024-07-24T22:39:27Z)
• Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priors [21.0128625037708]
  We present an innovative framework, divide-and-conquer posterior sampling. It reduces the approximation error associated with current techniques without the need for retraining. We demonstrate the versatility and effectiveness of our approach for a wide range of Bayesian inverse problems.
  arXiv  Detail & Related papers  (2024-03-18T01:47:24Z)
• Projection Regret: Reducing Background Bias for Novelty Detection via Diffusion Models [72.07462371883501]
  We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information. PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality. Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
  arXiv  Detail & Related papers  (2023-12-05T09:44:47Z)
• Refining Amortized Posterior Approximations using Gradient-Based Summary Statistics [0.9176056742068814]
  We present an iterative framework to improve the amortized approximations of posterior distributions in the context of inverse problems. We validate our method in a controlled setting by applying it to a stylized problem, and observe improved posterior approximations with each iteration.
  arXiv  Detail & Related papers  (2023-05-15T15:47:19Z)
• On Calibrating Diffusion Probabilistic Models [78.75538484265292]
  diffusion probabilistic models (DPMs) have achieved promising results in diverse generative tasks. We propose a simple way for calibrating an arbitrary pretrained DPM, with which the score matching loss can be reduced and the lower bounds of model likelihood can be increased. Our calibration method is performed only once and the resulting models can be used repeatedly for sampling.
  arXiv  Detail & Related papers  (2023-02-21T14:14:40Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Tracking disease outbreaks from sparse data with Bayesian inference [55.82986443159948]
  The COVID-19 pandemic provides new motivation for estimating the empirical rate of transmission during an outbreak. Standard methods struggle to accommodate the partial observability and sparse data common at finer scales. We propose a Bayesian framework which accommodates partial observability in a principled manner.
  arXiv  Detail & Related papers  (2020-09-12T20:37:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.