Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint

• URL: http://arxiv.org/abs/2403.10585v1
• Date: Fri, 15 Mar 2024 16:38:47 GMT
• Title: Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint
• Authors: Haoyue Tang, Tian Xie, Aosong Feng, Hanyu Wang, Chenyang Zhang, Yang Bai,
• Abstract summary: We leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning. To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG) Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks.
• Score: 21.22750301965104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Solving image inverse problems (e.g., super-resolution and inpainting) requires generating a high fidelity image that matches the given input (the low-resolution image or the masked image). By using the input image as guidance, we can leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning. To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG), a tractable computation method by viewing the intermediate noisy images as policies and the target image as the states selected by the policy. Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks, resulting into a higher image restoration quality on FFHQ, ImageNet and LSUN datasets.

Related papers

• Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
  Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance. Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs. In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
  arXiv  Detail & Related papers  (2023-11-20T08:27:56Z)
• Deep Richardson-Lucy Deconvolution for Low-Light Image Deblurring [48.80983873199214]
  We develop a data-driven approach to model the saturated pixels by a learned latent map. Based on the new model, the non-blind deblurring task can be formulated into a maximum a posterior (MAP) problem. To estimate high-quality deblurred images without amplified artifacts, we develop a prior estimation network.
  arXiv  Detail & Related papers  (2023-08-10T12:53:30Z)
• Image Restoration with Mean-Reverting Stochastic Differential Equations [9.245782611878752]
  This paper presents a differential equation (SDE) approach for general-purpose image restoration. By simulating the corresponding reverse-time SDE, we are able to restore the origin of the low-quality image. Experiments show that our proposed method achieves highly competitive performance in quantitative comparisons on image deraining, deblurring, and denoising.
  arXiv  Detail & Related papers  (2023-01-27T13:20:48Z)
• Denoising Diffusion Restoration Models [110.1244240726802]
  Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
  arXiv  Detail & Related papers  (2022-01-27T20:19:07Z)
• Inverse Problems Leveraging Pre-trained Contrastive Representations [88.70821497369785]
  We study a new family of inverse problems for recovering representations of corrupted data. We propose a supervised inversion method that uses a contrastive objective to obtain excellent representations for highly corrupted images. Our method outperforms end-to-end baselines even with a fraction of the labeled data in a wide range of forward operators.
  arXiv  Detail & Related papers  (2021-10-14T15:06:30Z)
• Joint Implicit Image Function for Guided Depth Super-Resolution [11.325235139023931]
  Guided depth super-resolution is a practical task where a low-resolution and noisy input depth map is restored to a high-resolution version. We take the form of a general image but use a novel Joint Implicit Image Function representation to learn both the weights and values. We demonstrate the effectiveness of our JIIF representation on guided depth super-resolution task, significantly outperforming state-of-the-art methods on three public benchmarks.
  arXiv  Detail & Related papers  (2021-07-19T09:42:18Z)
• Image Inpainting with Learnable Feature Imputation [8.293345261434943]
  A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. We propose (layer-wise) feature imputation of the missing input values to a convolution. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model.
  arXiv  Detail & Related papers  (2020-11-02T16:05:32Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.