Enhanced Denoising and Convergent Regularisation Using Tweedie Scaling

• URL: http://arxiv.org/abs/2503.05956v1
• Date: Fri, 07 Mar 2025 21:51:59 GMT
• Title: Enhanced Denoising and Convergent Regularisation Using Tweedie Scaling
• Authors: Naïl Khelifa, Ferdia Sherry, Carola-Bibiane Schönlieb,
• Abstract summary: This work introduces a novel scaling method that explicitly integrates and adjusts the strength of regularisation.<n>The scaling parameter enhances interpretability by reflecting the quality of the denoiser's learning process.<n>The proposed approach ensures that the resulting family of regularisations is provably stable and convergent.
• Score: 10.35768862376698
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The inherent ill-posed nature of image reconstruction problems, due to limitations in the physical acquisition process, is typically addressed by introducing a regularisation term that incorporates prior knowledge about the underlying image. The iterative framework of Plug-and-Play methods, specifically designed for tackling such inverse problems, achieves state-of-the-art performance by replacing the regularisation with a generic denoiser, which may be parametrised by a neural network architecture. However, these deep learning approaches suffer from a critical limitation: the absence of a control parameter to modulate the regularisation strength, which complicates the design of a convergent regularisation. To address this issue, this work introduces a novel scaling method that explicitly integrates and adjusts the strength of regularisation. The scaling parameter enhances interpretability by reflecting the quality of the denoiser's learning process, and also systematically improves its optimisation. Furthermore, the proposed approach ensures that the resulting family of regularisations is provably stable and convergent.

Related papers

• Structured Context Recomposition for Large Language Models Using Probabilistic Layer Realignment [0.0]
  This paper introduces a probabilistic layer realignment strategy that dynamically adjusts learned representations within transformer layers. It mitigates abrupt topic shifts and logical inconsistencies, particularly in scenarios where sequences exceed standard attention window constraints. While SCR incurs a moderate increase in processing time, memory overhead remains within feasible limits, making it suitable for practical deployment in autoregressive generative applications.
  arXiv  Detail & Related papers  (2025-01-29T12:46:42Z)
• Regularization for Adversarial Robust Learning [18.46110328123008]
  We develop a novel approach to adversarial training that integrates $phi$-divergence regularization into the distributionally robust risk function. This regularization brings a notable improvement in computation compared with the original formulation. We validate our proposed method in supervised learning, reinforcement learning, and contextual learning and showcase its state-of-the-art performance against various adversarial attacks.
  arXiv  Detail & Related papers  (2024-08-19T03:15:41Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Function-Space Regularization in Neural Networks: A Probabilistic Perspective [51.133793272222874]
  We show that we can derive a well-motivated regularization technique that allows explicitly encoding information about desired predictive functions into neural network training. We evaluate the utility of this regularization technique empirically and demonstrate that the proposed method leads to near-perfect semantic shift detection and highly-calibrated predictive uncertainty estimates.
  arXiv  Detail & Related papers  (2023-12-28T17:50:56Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Convex Latent-Optimized Adversarial Regularizers for Imaging Inverse Problems [8.33626757808923]
  We introduce Convex Latent-d Adrial Regularizers (CLEAR), a novel and interpretable data-driven paradigm. CLEAR represents a fusion of deep learning (DL) and variational regularization. Our method consistently outperforms conventional data-driven techniques and traditional regularization approaches.
  arXiv  Detail & Related papers  (2023-09-17T12:06:04Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization [69.80141512683254]
  We introduce Tabular Neural Gradient Orthogonalization and gradient (TANGOS) TANGOS is a novel framework for regularization in the tabular setting built on latent unit attributions. We demonstrate that our approach can lead to improved out-of-sample generalization performance, outperforming other popular regularization methods.
  arXiv  Detail & Related papers  (2023-03-09T18:57:13Z)
• Learning Sparsity-Promoting Regularizers using Bilevel Optimization [9.18465987536469]
  We present a method for supervised learning of sparsity-promoting regularizers for denoising signals and images. Experiments with structured 1D signals and natural images show that the proposed method can learn an operator that outperforms well-known regularizers.
  arXiv  Detail & Related papers  (2022-07-18T20:50:02Z)
• Rethinking Skip Connection with Layer Normalization in Transformers and ResNets [49.87919454950763]
  Skip connection is a widely-used technique to improve the performance of deep neural networks. In this work, we investigate how the scale factors in the effectiveness of the skip connection.
  arXiv  Detail & Related papers  (2021-05-15T11:44:49Z)
• Topological Regularization via Persistence-Sensitive Optimization [10.29838087001588]
  A key tool in machine learning and statistics, regularization relies on regularization to reduce overfitting. We propose a method that builds on persistence-sensitive simplification and translates required changes to the persistence diagram into changes on large subsets of the domain. This approach enables a faster and more precise topological regularization, the benefits of which we illustrate.
  arXiv  Detail & Related papers  (2020-11-10T18:19:43Z)
• Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations [52.493315075385325]
  We show that a family of regularizers, including weight decay, is ineffective at penalizing the intrinsic norms of weights for networks with homogeneous activation functions. We propose an improved regularizer that is invariant to weight scale shifting and thus effectively constrains the intrinsic norm of a neural network.
  arXiv  Detail & Related papers  (2020-08-07T02:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.