Optimizing Diffusion Priors in Image Reconstruction from a Single Observation

• URL: http://arxiv.org/abs/2604.21066v2
• Date: Fri, 24 Apr 2026 23:10:03 GMT
• Title: Optimizing Diffusion Priors in Image Reconstruction from a Single Observation
• Abstract summary: Current approaches to finetuning diffusion models rely on a large number of observations with varying forward operators.<n>We propose a method for tuning a prior from only a single observation by combining existing diffusion priors into a single product-of-experts prior.<n>We find that the evidence is often maximized by priors that extend beyond those trained on a single dataset.
• Score: 9.813689581505548
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While diffusion priors generate high-quality posterior samples across many inverse problems, they are often trained on limited training sets or purely simulated data, thus inheriting the errors and biases of these underlying sources. Current approaches to finetuning diffusion models rely on a large number of observations with varying forward operators, which can be difficult to collect for many applications, and thus lead to overfitting when the measurement set is small. We propose a method for tuning a prior from only a single observation by combining existing diffusion priors into a single product-of-experts prior and identifying the exponents that maximize the Bayesian evidence. We validate our method on real-world inverse problems, including black hole imaging, where the true prior is unknown a priori, and image deblurring with text-conditioned priors. We find that the evidence is often maximized by priors that extend beyond those trained on a single dataset. By generalizing the prior through exponent weighting, our approach enables posterior sampling from both tempered and combined diffusion models, yielding more flexible priors that improve the trustworthiness of the resulting posterior image distribution.

Related papers

• Sample-efficient evidence estimation of score based priors for model selection [9.813689581505548]
  We propose an estimator of the model evidence of a diffusion prior by integrating over the time-marginals of posterior sampling methods.<n>Our method leverages the large amount of intermediate samples naturally obtained during the reverse diffusion sampling process.<n>It is able to both select the correct diffusion model prior and diagnose prior misfit under different highly ill-conditioned, non-linear inverse problems.
  arXiv  Detail & Related papers  (2026-02-24T05:06:46Z)
• Test-Time Anchoring for Discrete Diffusion Posterior Sampling [38.507644561076894]
  Posterior sampling is a challenging problem for pretrained discrete diffusion foundation models.<n>We introduce Anchored Posterior Sampling (APS) for masked diffusion foundation models.<n>Our approach achieves state-of-the-art performance among discrete diffusion samplers across linear and nonlinear inverse problems.
  arXiv  Detail & Related papers  (2025-10-02T17:58:37Z)
• DPBridge: Latent Diffusion Bridge for Dense Prediction [49.1574468325115]
  We introduce DPBridge, the first latent diffusion bridge framework for dense prediction tasks.<n>Our method consistently achieves superior performance, demonstrating its effectiveness and capability generalization under different scenarios.
  arXiv  Detail & Related papers  (2024-12-29T15:50:34Z)
• Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems [12.482127049881026]
  We propose a novel approach to solve inverse problems with a diffusion prior from an amortized variational inference perspective. Our amortized inference learns a function that directly maps measurements to the implicit posterior distributions of corresponding clean data, enabling a single-step posterior sampling even for unseen measurements.
  arXiv  Detail & Related papers  (2024-07-23T02:14:18Z)
• Amortizing intractable inference in diffusion models for vision, language, and control [89.65631572949702]
  This paper studies amortized sampling of the posterior over data, $mathbfxsim prm post(mathbfx)propto p(mathbfx)r(mathbfx)$, in a model that consists of a diffusion generative model prior $p(mathbfx)$ and a black-box constraint or function $r(mathbfx)$.<n>We prove the correctness of a data-free learning objective, relative trajectory balance, for training a diffusion model that samples from
  arXiv  Detail & Related papers  (2024-05-31T16:18:46Z)
• Learning Diffusion Priors from Observations by Expectation Maximization [10.704978219090039]
  We present DiEM, a novel method for training diffusion models from incomplete and noisy observations only.<n>Unlike previous works, DiEM leads to proper diffusion models, which is crucial for downstream tasks.
  arXiv  Detail & Related papers  (2024-05-22T15:04:06Z)
• Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction [60.52716381465063]
  We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality pseudo-observations. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times.
  arXiv  Detail & Related papers  (2023-05-24T14:00:32Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)
• Score-Based Diffusion Models as Principled Priors for Inverse Imaging [46.19536250098105]
  We propose turning score-based diffusion models into principled image priors. We show how to sample from resulting posteriors by using this probability function for variational inference.
  arXiv  Detail & Related papers  (2023-04-23T21:05:59Z)
• Posterior samples of source galaxies in strong gravitational lenses with score-based priors [107.52670032376555]
  We use a score-based model to encode the prior for the inference of undistorted images of background galaxies. We show how the balance between the likelihood and the prior meet our expectations in an experiment with out-of-distribution data.
  arXiv  Detail & Related papers  (2022-11-07T19:00:42Z)
• Improving Diffusion Models for Inverse Problems using Manifold Constraints [55.91148172752894]
  We show that current solvers throw the sample path off the data manifold, and hence the error accumulates. To address this, we propose an additional correction term inspired by the manifold constraint. We show that our method is superior to the previous methods both theoretically and empirically.
  arXiv  Detail & Related papers  (2022-06-02T09:06:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.