From the Gradient-Step Denoiser to the Proximal Denoiser and their associated convergent Plug-and-Play algorithms

• URL: http://arxiv.org/abs/2509.09793v1
• Date: Thu, 11 Sep 2025 18:53:08 GMT
• Title: From the Gradient-Step Denoiser to the Proximal Denoiser and their associated convergent Plug-and-Play algorithms
• Authors: Vincent Herfeld, Baudouin Denis de Senneville, Arthur Leclaire, Nicolas Papadakis,
• Abstract summary: We analyze the Gradient-Step Denoiser and its usage in Plug-and-Play algorithms.<n>The Plug-and-Play paradigm of optimization algorithms uses off the shelf denoisers to replace a proximity operator or a gradient descent operator of an image prior.
• Score: 8.002203663865643
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we analyze the Gradient-Step Denoiser and its usage in Plug-and-Play algorithms. The Plug-and-Play paradigm of optimization algorithms uses off the shelf denoisers to replace a proximity operator or a gradient descent operator of an image prior. Usually this image prior is implicit and cannot be expressed, but the Gradient-Step Denoiser is trained to be exactly the gradient descent operator or the proximity operator of an explicit functional while preserving state-of-the-art denoising capabilities.

Related papers

• Gradient Normalization Provably Benefits Nonconvex SGD under Heavy-Tailed Noise [60.92029979853314]
  We investigate the roles of gradient normalization and clipping in ensuring the convergence of Gradient Descent (SGD) under heavy-tailed noise. Our work provides the first theoretical evidence demonstrating the benefits of gradient normalization in SGD under heavy-tailed noise. We introduce an accelerated SGD variant incorporating gradient normalization and clipping, further enhancing convergence rates under heavy-tailed noise.
  arXiv  Detail & Related papers  (2024-10-21T22:40:42Z)
• Adaptive Strategies in Non-convex Optimization [5.279475826661643]
  An algorithm is said to be adaptive to a certain parameter if it does not need a priori knowledge of such a parameter. This dissertation presents our work on adaptive algorithms in three scenarios.
  arXiv  Detail & Related papers  (2023-06-17T06:52:05Z)
• A Unified Framework for Implicit Sinkhorn Differentiation [58.56866763433335]
  We propose an algorithm that obtains analytical gradients of a Sinkhorn layer via implicit differentiation. We show that it is computationally more efficient, particularly when resources like GPU memory are scarce.
  arXiv  Detail & Related papers  (2022-05-13T14:45:31Z)
• Learned Gradient of a Regularizer for Plug-and-Play Gradient Descent [37.41458921829744]
  The Plug-and-Play framework allows integrating advanced image denoising priors into algorithms. Regularization by Denoising (RED) algorithms are two examples of methods that made a breakthrough in image restoration. We show that it is possible to train a denoiser along with a network that corresponds to the gradient of its regularizer.
  arXiv  Detail & Related papers  (2022-04-29T08:33:33Z)
• Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization [7.0226402509856225]
  Plug-and-Play () methods solve ill proximal-posed inverse problems through algorithms by replacing a neural network operator by a denoising operator. We show that this denoiser actually correspond to a gradient function.
  arXiv  Detail & Related papers  (2022-01-31T14:05:20Z)
• Gradient Step Denoiser for convergent Plug-and-Play [5.629161809575015]
  Plug-and-Play methods can lead to tremendous visual performance for various image problems. We propose new type of Plug-and-Play methods, based on half-quadratic descent. Experiments show that it is possible to learn such a deep denoiser while not compromising the performance.
  arXiv  Detail & Related papers  (2021-10-07T07:11:48Z)
• Preconditioned Plug-and-Play ADMM with Locally Adjustable Denoiser for Image Restoration [54.23646128082018]
  We extend the concept of plug-and-play optimization to use denoisers that can be parameterized for non-constant noise variance. We show that our pixel-wise adjustable denoiser, along with a suitable preconditioning strategy, can further improve the plug-and-play ADMM approach for several applications.
  arXiv  Detail & Related papers  (2021-10-01T15:46:35Z)
• An Accelerated Variance-Reduced Conditional Gradient Sliding Algorithm for First-order and Zeroth-order Optimization [111.24899593052851]
  Conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community. ARCS is the first zeroth-order conditional gradient sliding type algorithms solving convex problems in zeroth-order optimization. In first-order optimization, the convergence results of ARCS substantially outperform previous algorithms in terms of the number of gradient query oracle.
  arXiv  Detail & Related papers  (2021-09-18T07:08:11Z)
• Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering [53.523517926927894]
  We explore the use of exact per-sample Hessian-vector products and gradients to construct self-tuning quadratics. We prove that our model-based procedure converges in noisy gradient setting. This is an interesting step for constructing self-tuning quadratics.
  arXiv  Detail & Related papers  (2020-11-09T22:07:30Z)
• Variance Reduction with Sparse Gradients [82.41780420431205]
  Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients. We introduce a new sparsity operator: The random-top-k operator. Our algorithm consistently outperforms SpiderBoost on various tasks including image classification, natural language processing, and sparse matrix factorization.
  arXiv  Detail & Related papers  (2020-01-27T08:23:58Z)
• Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets [71.05306664267832]
  Adaptive algorithms perform gradient updates using the history of gradients and are ubiquitous in training deep neural networks. In this paper we analyze a variant of OptimisticOA algorithm for nonconcave minmax problems. Our experiments show that adaptive GAN non-adaptive gradient algorithms can be observed empirically.
  arXiv  Detail & Related papers  (2019-12-26T22:10:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.