A Regularized Conditional GAN for Posterior Sampling in Image Recovery Problems

• URL: http://arxiv.org/abs/2210.13389v5
• Date: Fri, 27 Oct 2023 13:52:56 GMT
• Title: A Regularized Conditional GAN for Posterior Sampling in Image Recovery Problems
• Authors: Matthew Bendel, Rizwan Ahmad, and Philip Schniter
• Abstract summary: In image recovery problems, one seeks to infer an image from distorted, incomplete, and/or noise-corrupted measurements. We propose a regularized conditional Wasserstein GAN that generates dozens of high-quality posterior samples per second. Our method produces state-of-the-art posterior samples in both multicoil MRI and large-scale inpainting applications.
• Score: 11.393603788068777
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In image recovery problems, one seeks to infer an image from distorted, incomplete, and/or noise-corrupted measurements. Such problems arise in magnetic resonance imaging (MRI), computed tomography, deblurring, super-resolution, inpainting, phase retrieval, image-to-image translation, and other applications. Given a training set of signal/measurement pairs, we seek to do more than just produce one good image estimate. Rather, we aim to rapidly and accurately sample from the posterior distribution. To do this, we propose a regularized conditional Wasserstein GAN that generates dozens of high-quality posterior samples per second. Our regularization comprises an $\ell_1$ penalty and an adaptively weighted standard-deviation reward. Using quantitative evaluation metrics like conditional Fr\'{e}chet inception distance, we demonstrate that our method produces state-of-the-art posterior samples in both multicoil MRI and large-scale inpainting applications. The code for our model can be found here: https://github.com/matt-bendel/rcGAN

Related papers

• pcaGAN: Improving Posterior-Sampling cGANs via Principal Component Regularization [11.393603788068777]
  In ill-posed imaging inverse problems, there can exist many hypotheses that fit both the observed measurements and prior knowledge of the true image. We propose a fast and accurate posterior-sampling conditional generative adversarial network (cGAN) that, through a novel form of regularization, aims for correctness in the posterior mean.
  arXiv  Detail & Related papers  (2024-11-01T14:09:28Z)
• Task-Driven Uncertainty Quantification in Inverse Problems via Conformal Prediction [11.393603788068777]
  In imaging inverse problems, one seeks to recover an image from missing/corrupted measurements. Motivated by applications where the recovered image is used for a downstream task, we propose a task-centered approach to uncertainty quantification.
  arXiv  Detail & Related papers  (2024-05-28T18:49:17Z)
• Autoregressive Image Diffusion: Generation of Image Sequence and Application in MRI [2.0318411357438086]
  Generative models learn image distributions and can be used to reconstruct high-quality images from undersampled k-space data. We present the autoregressive image diffusion (AID) model for image sequences and use it to sample the posterior for accelerated MRI reconstruction. The results show that the AID model can robustly generate sequentially coherent image sequences.
  arXiv  Detail & Related papers  (2024-05-23T08:57:10Z)
• Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models trained on Corrupted Data [56.81246107125692]
  Ambient Diffusion Posterior Sampling (A-DPS) is a generative model pre-trained on one type of corruption. We show that A-DPS can sometimes outperform models trained on clean data for several image restoration tasks in both speed and performance. We extend the Ambient Diffusion framework to train MRI models with access only to Fourier subsampled multi-coil MRI measurements.
  arXiv  Detail & Related papers  (2024-03-13T17:28:20Z)
• Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
  Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance. Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs. In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
  arXiv  Detail & Related papers  (2023-11-20T08:27:56Z)
• A Conditional Normalizing Flow for Accelerated Multi-Coil MR Imaging [12.31503281925152]
  We develop a novel conditional normalizing flow (CNF) that infers the signal component in the measurement operator's nullspace, which is later combined with measured data to form complete images. Using fastMRI brain and knee data, we demonstrate fast inference and accuracy that surpasses recent posterior sampling techniques for MRI.
  arXiv  Detail & Related papers  (2023-06-02T15:49:26Z)
• One Sample Diffusion Model in Projection Domain for Low-Dose CT Imaging [10.797632196651731]
  Low-dose computed tomography (CT) plays a significant role in reducing the radiation risk in clinical applications. With the rapid development and wide application of deep learning, it has brought new directions for the development of low-dose CT imaging algorithms. We propose a fully unsupervised one sample diffusion model (OSDM)in projection domain for low-dose CT reconstruction. The results prove that OSDM is practical and effective model for reducing the artifacts and preserving the image quality.
  arXiv  Detail & Related papers  (2022-12-07T13:39:23Z)
• Posterior samples of source galaxies in strong gravitational lenses with score-based priors [107.52670032376555]
  We use a score-based model to encode the prior for the inference of undistorted images of background galaxies. We show how the balance between the likelihood and the prior meet our expectations in an experiment with out-of-distribution data.
  arXiv  Detail & Related papers  (2022-11-07T19:00:42Z)
• Deep Unfolded Recovery of Sub-Nyquist Sampled Ultrasound Image [94.42139459221784]
  We propose a reconstruction method from sub-Nyquist samples in the time and spatial domain, that is based on unfolding the ISTA algorithm. Our method allows reducing the number of array elements, sampling rate, and computational time while ensuring high quality imaging performance.
  arXiv  Detail & Related papers  (2021-03-01T19:19:38Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• Improved Slice-wise Tumour Detection in Brain MRIs by Computing Dissimilarities between Latent Representations [68.8204255655161]
  Anomaly detection for Magnetic Resonance Images (MRIs) can be solved with unsupervised methods. We have proposed a slice-wise semi-supervised method for tumour detection based on the computation of a dissimilarity function in the latent space of a Variational AutoEncoder. We show that by training the models on higher resolution images and by improving the quality of the reconstructions, we obtain results which are comparable with different baselines.
  arXiv  Detail & Related papers  (2020-07-24T14:02:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.