Equivariant plug-and-play image reconstruction

• URL: http://arxiv.org/abs/2312.01831v2
• Date: Thu, 23 May 2024 15:52:57 GMT
• Title: Equivariant plug-and-play image reconstruction
• Authors: Matthieu Terris, Thomas Moreau, Nelly Pustelnik, Julian Tachella,
• Abstract summary: Plug-and-play algorithms can leverage powerful pre-trained denoisers to solve inverse imaging problems. We show that enforcing equivariance to certain groups of transformations on the denoiser improves the stability of the algorithm as well as its reconstruction quality. Experiments on multiple imaging modalities and denoising networks show that the equivariant plug-and-play algorithm improves both the reconstruction performance and the stability compared to their non-equivariant counterparts.
• Score: 10.781078029828473
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Plug-and-play algorithms constitute a popular framework for solving inverse imaging problems that rely on the implicit definition of an image prior via a denoiser. These algorithms can leverage powerful pre-trained denoisers to solve a wide range of imaging tasks, circumventing the necessity to train models on a per-task basis. Unfortunately, plug-and-play methods often show unstable behaviors, hampering their promise of versatility and leading to suboptimal quality of reconstructed images. In this work, we show that enforcing equivariance to certain groups of transformations (rotations, reflections, and/or translations) on the denoiser strongly improves the stability of the algorithm as well as its reconstruction quality. We provide a theoretical analysis that illustrates the role of equivariance on better performance and stability. We present a simple algorithm that enforces equivariance on any existing denoiser by simply applying a random transformation to the input of the denoiser and the inverse transformation to the output at each iteration of the algorithm. Experiments on multiple imaging modalities and denoising networks show that the equivariant plug-and-play algorithm improves both the reconstruction performance and the stability compared to their non-equivariant counterparts.

Related papers

• Iteratively Refined Image Reconstruction with Learned Attentive Regularizers [14.93489065234423]
  We propose a regularization scheme for image reconstruction that leverages the power of deep learning. Our scheme is interpretable because it corresponds to the minimization of a series of convex problems. We offer a promising balance between interpretability, theoretical guarantees, reliability, and performance.
  arXiv  Detail & Related papers  (2024-07-09T07:22:48Z)
• ReNoise: Real Image Inversion Through Iterative Noising [62.96073631599749]
  We introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models.
  arXiv  Detail & Related papers  (2024-03-21T17:52:08Z)
• Plug-and-Play image restoration with Stochastic deNOising REgularization [8.678250057211368]
  We propose a new framework called deNOising REgularization (SNORE) SNORE applies the denoiser only to images with noise of the adequate level. It is based on an explicit regularization, which leads to a descent to solve inverse problems.
  arXiv  Detail & Related papers  (2024-02-01T18:05:47Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)
• Denoising Diffusion Restoration Models [110.1244240726802]
  Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
  arXiv  Detail & Related papers  (2022-01-27T20:19:07Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• Learned convex regularizers for inverse problems [3.294199808987679]
  We propose to learn a data-adaptive input- neural network (ICNN) as a regularizer for inverse problems. We prove the existence of a sub-gradient-based algorithm that leads to a monotonically decreasing error in the parameter space with iterations. We show that the proposed convex regularizer is at least competitive with and sometimes superior to state-of-the-art data-driven techniques for inverse problems.
  arXiv  Detail & Related papers  (2020-08-06T18:58:35Z)
• Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction [4.2476585678737395]
  We propose a general learning based framework for solving nonsmooth non image reconstruction problems. We show that the proposed is-efficient convergence state-of-the-art methods in an image problems in training.
  arXiv  Detail & Related papers  (2020-07-22T07:59:07Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)
• Fully Unsupervised Diversity Denoising with Convolutional Variational Autoencoders [81.30960319178725]
  We propose DivNoising, a denoising approach based on fully convolutional variational autoencoders (VAEs) First we introduce a principled way of formulating the unsupervised denoising problem within the VAE framework by explicitly incorporating imaging noise models into the decoder. We show that such a noise model can either be measured, bootstrapped from noisy data, or co-learned during training.
  arXiv  Detail & Related papers  (2020-06-10T21:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.