Plug-and-Play Deep Energy Model for Inverse problems

• URL: http://arxiv.org/abs/2302.11570v1
• Date: Wed, 15 Feb 2023 09:44:45 GMT
• Title: Plug-and-Play Deep Energy Model for Inverse problems
• Authors: Jyothi Rikabh Chand, Mathews Jacob
• Abstract summary: We introduce a novel energy formulation for Plug- and-Play (CNN) image recovery. The proposed model offers algorithms with convergence guarantees, even when the learned score model is not a contraction model.
• Score: 18.047694351309204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a novel energy formulation for Plug- and-Play (PnP) image recovery. Traditional PnP methods that use a convolutional neural network (CNN) do not have an energy based formulation. The primary focus of this work is to introduce an energy-based PnP formulation, which relies on a CNN that learns the log of the image prior from training data. The score function is evaluated as the gradient of the energy model, which resembles a UNET with shared encoder and decoder weights. The proposed score function is thus constrained to a conservative vector field, which is the key difference with classical PnP models. The energy-based formulation offers algorithms with convergence guarantees, even when the learned score model is not a contraction. The relaxation of the contraction constraint allows the proposed model to learn more complex priors, thus offering improved performance over traditional PnP schemes. Our experiments in magnetic resonance image reconstruction demonstrates the improved performance offered by the proposed energy model over traditional PnP methods.

Related papers

• MAP Image Recovery with Guarantees using Locally Convex Multi-Scale Energy (LC-MUSE) Model [12.218356507147583]
  We propose a multi-scale deep energy model that is strongly convex in the local neighbourhood around the data manifold. We use the learned energy model in image-based inverse problems, where the formulation offers several desirable properties. In the context of parallel Magnetic Resonance (MR) image reconstruction, we show that the proposed method performs better than the state-of-the-art convex regularizers.
  arXiv  Detail & Related papers  (2025-02-05T16:00:55Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval. A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed. The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• vHeat: Building Vision Models upon Heat Conduction [63.00030330898876]
  vHeat is a novel vision backbone model that simultaneously achieves both high computational efficiency and global receptive field. The essential idea is to conceptualize image patches as heat sources and model the calculation of their correlations as the diffusion of thermal energy.
  arXiv  Detail & Related papers  (2024-05-26T12:58:04Z)
• Generative Plug and Play: Posterior Sampling for Inverse Problems [4.417934991211913]
  Plug-Play (and) has become a popular method for reconstructing images using a framework consisting of a forward and prior model. We present experimental simulations using the well-known BM3D denoiser.
  arXiv  Detail & Related papers  (2023-06-12T16:49:08Z)
• A Neural Network Implementation for Free Energy Principle [3.7937771805690392]
  The free energy principle (FEP) has been widely applied to account for various problems in fields such as cognitive science, neuroscience, social interaction, and hermeneutics. This paper gives a preliminary attempt at bridging FEP and machine learning, via a classical neural network model, the Helmholtz machine. Although the Helmholtz machine is not temporal, it gives an ideal parallel to the vanilla FEP and the hierarchical model of the brain, under which the active inference and predictive coding could be formulated coherently.
  arXiv  Detail & Related papers  (2023-06-11T22:14:21Z)
• Energy Transformer [64.22957136952725]
  Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory. We propose a novel architecture, called the Energy Transformer (or ET for short), that uses a sequence of attention layers that are purposely designed to minimize a specifically engineered energy function.
  arXiv  Detail & Related papers  (2023-02-14T18:51:22Z)
• Online Deep Equilibrium Learning for Regularization by Denoising [20.331171081002957]
  Plug-and-Play Equilibrium Priors (memory) and Regularization by Denoising (RED) are widely-used frameworks for solving inverse imaging problems by computing fixed-points. We propose ODER as a new strategy for improving the efficiency of DEQ/RED on the total number of measurements. Our numerical results suggest the potential improvements in training/testing complexity due to ODER on three distinct imaging applications.
  arXiv  Detail & Related papers  (2022-05-25T21:06:22Z)
• An Energy-Based Prior for Generative Saliency [62.79775297611203]
  We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution. With the generative saliency model, we can obtain a pixel-wise uncertainty map from an image, indicating model confidence in the saliency prediction. Experimental results show that our generative saliency model with an energy-based prior can achieve not only accurate saliency predictions but also reliable uncertainty maps consistent with human perception.
  arXiv  Detail & Related papers  (2022-04-19T10:51:00Z)
• Dual Training of Energy-Based Models with Overparametrized Shallow Neural Networks [41.702175127106784]
  Energy-based models (EBMs) are generative models that are usually trained via maximum likelihood estimation. We propose a dual formulation of an EBMs algorithm in which the particles are sometimes restarted at random samples drawn from the data set, and show that performing these restarts corresponds to a score every step. These results are illustrated in simple numerical experiments.
  arXiv  Detail & Related papers  (2021-07-11T21:43:18Z)
• A Deep-Unfolded Reference-Based RPCA Network For Video Foreground-Background Separation [86.35434065681925]
  This paper proposes a new deep-unfolding-based network design for the problem of Robust Principal Component Analysis (RPCA) Unlike existing designs, our approach focuses on modeling the temporal correlation between the sparse representations of consecutive video frames. Experimentation using the moving MNIST dataset shows that the proposed network outperforms a recently proposed state-of-the-art RPCA network in the task of video foreground-background separation.
  arXiv  Detail & Related papers  (2020-10-02T11:40:09Z)
• Targeted free energy estimation via learned mappings [66.20146549150475]
  Free energy perturbation (FEP) was proposed by Zwanzig more than six decades ago as a method to estimate free energy differences. FEP suffers from a severe limitation: the requirement of sufficient overlap between distributions. One strategy to mitigate this problem, called Targeted Free Energy Perturbation, uses a high-dimensional mapping in configuration space to increase overlap.
  arXiv  Detail & Related papers  (2020-02-12T11:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.