Iteratively Refined Image Reconstruction with Learned Attentive Regularizers

• URL: http://arxiv.org/abs/2407.06608v1
• Date: Tue, 9 Jul 2024 07:22:48 GMT
• Title: Iteratively Refined Image Reconstruction with Learned Attentive Regularizers
• Authors: Mehrsa Pourya, Sebastian Neumayer, Michael Unser,
• Abstract summary: We propose a regularization scheme for image reconstruction that leverages the power of deep learning. Our scheme is interpretable because it corresponds to the minimization of a series of convex problems. We offer a promising balance between interpretability, theoretical guarantees, reliability, and performance.
• Score: 14.93489065234423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a regularization scheme for image reconstruction that leverages the power of deep learning while hinging on classic sparsity-promoting models. Many deep-learning-based models are hard to interpret and cumbersome to analyze theoretically. In contrast, our scheme is interpretable because it corresponds to the minimization of a series of convex problems. For each problem in the series, a mask is generated based on the previous solution to refine the regularization strength spatially. In this way, the model becomes progressively attentive to the image structure. For the underlying update operator, we prove the existence of a fixed point. As a special case, we investigate a mask generator for which the fixed-point iterations converge to a critical point of an explicit energy functional. In our experiments, we match the performance of state-of-the-art learned variational models for the solution of inverse problems. Additionally, we offer a promising balance between interpretability, theoretical guarantees, reliability, and performance.

Related papers

• Convergence Properties of Score-Based Models for Linear Inverse Problems Using Graduated Optimisation [44.99833362998488]
  We show that score-based generative models (SGMs) can be used to solve inverse problems. We show that we are able to recover high-Ms images, independent of the initial value. The source is publicly available on GitHub.
  arXiv  Detail & Related papers  (2024-04-29T13:47:59Z)
• Convex Latent-Optimized Adversarial Regularizers for Imaging Inverse Problems [8.33626757808923]
  We introduce Convex Latent-d Adrial Regularizers (CLEAR), a novel and interpretable data-driven paradigm. CLEAR represents a fusion of deep learning (DL) and variational regularization. Our method consistently outperforms conventional data-driven techniques and traditional regularization approaches.
  arXiv  Detail & Related papers  (2023-09-17T12:06:04Z)
• Regularized Vector Quantization for Tokenized Image Synthesis [126.96880843754066]
  Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while quantization suffers from low codebook utilization and reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate perturbed above issues effectively by applying regularization from two perspectives.
  arXiv  Detail & Related papers  (2023-03-11T15:20:54Z)
• Regularization via deep generative models: an analysis point of view [8.818465117061205]
  This paper proposes a new way of regularizing an inverse problem in imaging (e.g., deblurring or inpainting) by means of a deep generative neural network. In many cases our technique achieves a clear improvement of the performance and seems to be more robust.
  arXiv  Detail & Related papers  (2021-01-21T15:04:57Z)
• Counterfactual Generative Networks [59.080843365828756]
  We propose to decompose the image generation process into independent causal mechanisms that we train without direct supervision. By exploiting appropriate inductive biases, these mechanisms disentangle object shape, object texture, and background. We show that the counterfactual images can improve out-of-distribution with a marginal drop in performance on the original classification task.
  arXiv  Detail & Related papers  (2021-01-15T10:23:12Z)
• Shared Prior Learning of Energy-Based Models for Image Reconstruction [69.72364451042922]
  We propose a novel learning-based framework for image reconstruction particularly designed for training without ground truth data. In the absence of ground truth data, we change the loss functional to a patch-based Wasserstein functional. In shared prior learning, both aforementioned optimal control problems are optimized simultaneously with shared learned parameters of the regularizer.
  arXiv  Detail & Related papers  (2020-11-12T17:56:05Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• Learned convex regularizers for inverse problems [3.294199808987679]
  We propose to learn a data-adaptive input- neural network (ICNN) as a regularizer for inverse problems. We prove the existence of a sub-gradient-based algorithm that leads to a monotonically decreasing error in the parameter space with iterations. We show that the proposed convex regularizer is at least competitive with and sometimes superior to state-of-the-art data-driven techniques for inverse problems.
  arXiv  Detail & Related papers  (2020-08-06T18:58:35Z)
• Total Deep Variation: A Stable Regularizer for Inverse Problems [71.90933869570914]
  We introduce the data-driven general-purpose total deep variation regularizer. In its core, a convolutional neural network extracts local features on multiple scales and in successive blocks. We achieve state-of-the-art results for numerous imaging tasks.
  arXiv  Detail & Related papers  (2020-06-15T21:54:15Z)
• Total Deep Variation for Linear Inverse Problems [71.90933869570914]
  We propose a novel learnable general-purpose regularizer exploiting recent architectural design patterns from deep learning. We show state-of-the-art performance for classical image restoration and medical image reconstruction problems.
  arXiv  Detail & Related papers  (2020-01-14T19:01:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.