PnP-Flow: Plug-and-Play Image Restoration with Flow Matching

• URL: http://arxiv.org/abs/2410.02423v1
• Date: Thu, 3 Oct 2024 12:13:56 GMT
• Title: PnP-Flow: Plug-and-Play Image Restoration with Flow Matching
• Authors: Ségolène Martin, Anne Gagneux, Paul Hagemann, Gabriele Steidl,
• Abstract summary: We introduce Plug-and-Play Flow Matching, an algorithm for solving inverse imaging problems. We evaluate its performance on denoising, superresolution, and inpainting tasks.
• Score: 2.749898166276854
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: In this paper, we introduce Plug-and-Play (PnP) Flow Matching, an algorithm for solving imaging inverse problems. PnP methods leverage the strength of pre-trained denoisers, often deep neural networks, by integrating them in optimization schemes. While they achieve state-of-the-art performance on various inverse problems in imaging, PnP approaches face inherent limitations on more generative tasks like inpainting. On the other hand, generative models such as Flow Matching pushed the boundary in image sampling yet lack a clear method for efficient use in image restoration. We propose to combine the PnP framework with Flow Matching (FM) by defining a time-dependent denoiser using a pre-trained FM model. Our algorithm alternates between gradient descent steps on the data-fidelity term, reprojections onto the learned FM path, and denoising. Notably, our method is computationally efficient and memory-friendly, as it avoids backpropagation through ODEs and trace computations. We evaluate its performance on denoising, super-resolution, deblurring, and inpainting tasks, demonstrating superior results compared to existing PnP algorithms and Flow Matching based state-of-the-art methods.

Related papers

• Plug-and-Play image restoration with Stochastic deNOising REgularization [8.678250057211368]
  We propose a new framework called deNOising REgularization (SNORE) SNORE applies the denoiser only to images with noise of the adequate level. It is based on an explicit regularization, which leads to a descent to solve inverse problems.
  arXiv  Detail & Related papers  (2024-02-01T18:05:47Z)
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
  Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation. We present GradPaint, which steers the generation towards a globally coherent image. We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-09-18T09:36:24Z)
• Poisson-Gaussian Holographic Phase Retrieval with Score-based Image Prior [19.231581775644617]
  We propose a new algorithm called "AWFS" that uses the accelerated Wirtinger flow (AWF) with a score function as generative prior. We calculate the gradient of the log-likelihood function for PR and determine the Lipschitz constant. We provide theoretical analysis that establishes a critical-point convergence guarantee for the proposed algorithm.
  arXiv  Detail & Related papers  (2023-05-12T18:08:47Z)
• Cross-boosting of WNNM Image Denoising method by Directional Wavelet Packets [2.7648976108201815]
  The paper presents an image denoising scheme by combining a method that is based on directional quasi-analytic wavelet packets (qWPs) with the state-of-the-art Weighted Nuclear Norm Minimization (WNNM) denoising algorithm. The proposed methodology couples the qWPdn capabilities to capture edges and fine texture patterns even in the severely corrupted images.
  arXiv  Detail & Related papers  (2022-06-09T11:37:46Z)
• Denoising Diffusion Restoration Models [110.1244240726802]
  Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
  arXiv  Detail & Related papers  (2022-01-27T20:19:07Z)
• Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
  We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms. Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-11-27T12:12:57Z)
• Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoising [104.59305271099967]
  We present a pixel aggregation network and learn the pixel sampling and averaging strategies for image denoising. We develop a pixel aggregation network for video denoising to sample pixels across the spatial-temporal space. Our method is able to solve the misalignment issues caused by large motion in dynamic scenes.
  arXiv  Detail & Related papers  (2021-01-26T13:00:46Z)
• TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems [22.239477171296056]
  Plug-and-Play (MM) is a non- optimization framework that combines numerical algorithms, for example, with advanced denoising priors. We discuss several practical considerations of more denoisers, which together with our learned strategies are state-of-the-art results.
  arXiv  Detail & Related papers  (2020-11-18T14:19:30Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• A Fast Stochastic Plug-and-Play ADMM for Imaging Inverse Problems [5.025654873456756]
  We propose an efficient plug-and-play ( inverse problems) algorithm for imaging applications. Our results demonstrate effectiveness of our approach compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-20T18:03:52Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.