Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems

• URL: http://arxiv.org/abs/2409.20175v1
• Date: Mon, 30 Sep 2024 10:36:41 GMT
• Title: Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems
• Authors: Hongkai Zheng, Wenda Chu, Austin Wang, Nikola Kovachki, Ricardo Baptista, Yisong Yue,
• Abstract summary: In inverse problems, it is increasingly popular to use pre-trained diffusion models as plug-and-play priors. Most existing methods rely on privileged information such as derivative, pseudo-inverse, or full knowledge about the forward model. We propose Ensemble Kalman Diffusion Guidance (EnKG) for diffusion models, a derivative-free approach that can solve inverse problems by only accessing forward model evaluations and a pre-trained diffusion model prior.
• Score: 21.95946380639509
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: When solving inverse problems, it is increasingly popular to use pre-trained diffusion models as plug-and-play priors. This framework can accommodate different forward models without re-training while preserving the generative capability of diffusion models. Despite their success in many imaging inverse problems, most existing methods rely on privileged information such as derivative, pseudo-inverse, or full knowledge about the forward model. This reliance poses a substantial limitation that restricts their use in a wide range of problems where such information is unavailable, such as in many scientific applications. To address this issue, we propose Ensemble Kalman Diffusion Guidance (EnKG) for diffusion models, a derivative-free approach that can solve inverse problems by only accessing forward model evaluations and a pre-trained diffusion model prior. We study the empirical effectiveness of our method across various inverse problems, including scientific settings such as inferring fluid flows and astronomical objects, which are highly non-linear inverse problems that often only permit black-box access to the forward model.

Related papers

• G2D2: Gradient-guided Discrete Diffusion for image inverse problem solving [55.185588994883226]
  This paper presents a novel method for addressing linear inverse problems by leveraging image-generation models based on discrete diffusion as priors. To the best of our knowledge, this is the first approach to use discrete diffusion model-based priors for solving image inverse problems.
  arXiv  Detail & Related papers  (2024-10-09T06:18:25Z)
• Diffusion State-Guided Projected Gradient for Inverse Problems [82.24625224110099]
  We propose Diffusion State-Guided Projected Gradient (DiffStateGrad) for inverse problems. DiffStateGrad projects the measurement gradient onto a subspace that is a low-rank approximation of an intermediate state of the diffusion process. We highlight that DiffStateGrad improves the robustness of diffusion models in terms of the choice of measurement guidance step size and noise.
  arXiv  Detail & Related papers  (2024-10-04T14:26:54Z)
• A Survey on Diffusion Models for Inverse Problems [110.6628926886398]
  We provide an overview of methods that utilize pre-trained diffusion models to solve inverse problems without requiring further training. We discuss specific challenges and potential solutions associated with using latent diffusion models for inverse problems.
  arXiv  Detail & Related papers  (2024-09-30T17:34:01Z)
• Learning Diffusion Priors from Observations by Expectation Maximization [6.224769485481242]
  We present a novel method based on the expectation-maximization algorithm for training diffusion models from incomplete and noisy observations only. As part of our method, we propose and motivate an improved posterior sampling scheme for unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-05-22T15:04:06Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• Training-free Linear Image Inverses via Flows [17.291903204982326]
  We propose a training-free method for solving linear inverse problems by using pretrained flow models. Our approach requires no problem-specific tuning across an extensive suite of noisy linear inverse problems on high-dimensional datasets.
  arXiv  Detail & Related papers  (2023-09-25T22:13:16Z)
• Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency [7.671153315762146]
  Training diffusion models in the pixel space are both data-intensive and computationally demanding. Latent diffusion models, which operate in a much lower-dimensional space, offer a solution to these challenges. We propose textitReSample, an algorithm that can solve general inverse problems with pre-trained latent diffusion models.
  arXiv  Detail & Related papers  (2023-07-16T18:42:01Z)
• Eliminating Lipschitz Singularities in Diffusion Models [51.806899946775076]
  We show that diffusion models frequently exhibit the infinite Lipschitz near the zero point of timesteps. This poses a threat to the stability and accuracy of the diffusion process, which relies on integral operations. We propose a novel approach, dubbed E-TSDM, which eliminates the Lipschitz of the diffusion model near zero.
  arXiv  Detail & Related papers  (2023-06-20T03:05:28Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.