Prior Image-Constrained Reconstruction using Style-Based Generative Models

• URL: http://arxiv.org/abs/2102.12525v2
• Date: Mon, 14 Jun 2021 23:22:13 GMT
• Title: Prior Image-Constrained Reconstruction using Style-Based Generative Models
• Authors: Varun A. Kelkar, Mark A. Anastasio
• Abstract summary: We propose a framework for estimating an object of interest that is semantically related to a known prior image. An optimization problem is formulated in the disentangled latent space of a style-based generative model. Semantically meaningful constraints are imposed using the disentangled latent representation of the prior image.
• Score: 15.757204774959366
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Obtaining a useful estimate of an object from highly incomplete imaging measurements remains a holy grail of imaging science. Deep learning methods have shown promise in learning object priors or constraints to improve the conditioning of an ill-posed imaging inverse problem. In this study, a framework for estimating an object of interest that is semantically related to a known prior image, is proposed. An optimization problem is formulated in the disentangled latent space of a style-based generative model, and semantically meaningful constraints are imposed using the disentangled latent representation of the prior image. Stable recovery from incomplete measurements with the help of a prior image is theoretically analyzed. Numerical experiments demonstrating the superior performance of our approach as compared to related methods are presented.

Related papers

• Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation [50.16004183320537]
  We describe a method for recovering the irradiance underlying a collection of images corrupted by atmospheric turbulence. We select one of the images as a reference, and model the deformation in this image by the aggregation of the optical flow from it to other images. We achieve state-of-the-art performance despite its simplicity.
  arXiv  Detail & Related papers  (2024-05-06T17:39:53Z)
• Unsupervised Training of Convex Regularizers using Maximum Likelihood Estimation [12.625383613718636]
  We propose an unsupervised approach using maximum marginal likelihood estimation to train a convex neural network-based image regularization term directly on noisy measurements. Experiments demonstrate that the proposed method produces priors that are near competitive when compared to the analogous supervised training method for various image corruption operators.
  arXiv  Detail & Related papers  (2024-04-08T12:27:00Z)
• Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior [7.155937118886449]
  We consider ill-posed inverse imaging problems in which one aims for a clean image posterior given incomplete or noisy measurements. Recent work turned score-based diffusion models into principled priors for solving ill-posed imaging problems. Our proposed surrogate prior is based on the evidence lower bound of a score-based diffusion model.
  arXiv  Detail & Related papers  (2023-09-05T04:55:10Z)
• Disentangled Pre-training for Image Matting [74.10407744483526]
  Image matting requires high-quality pixel-level human annotations to support the training of a deep model. We propose a self-supervised pre-training approach that can leverage infinite numbers of data to boost the matting performance.
  arXiv  Detail & Related papers  (2023-04-03T08:16:02Z)
• TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimation [55.94900327396771]
  We introduce neural texture learning for 6D object pose estimation from synthetic data. We learn to predict realistic texture of objects from real image collections. We learn pose estimation from pixel-perfect synthetic data.
  arXiv  Detail & Related papers  (2022-12-25T13:36:32Z)
• Object-centric and memory-guided normality reconstruction for video anomaly detection [56.64792194894702]
  This paper addresses anomaly detection problem for videosurveillance. Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy. Our model learns object-centric normal patterns without seeing anomalous samples during training.
  arXiv  Detail & Related papers  (2022-03-07T19:28:39Z)
• Prior image-based medical image reconstruction using a style-based generative adversarial network [15.757204774959366]
  This work proposes to use a style-based generative adversarial network (StyleGAN) to constrain an image reconstruction problem. An optimization problem is formulated in the intermediate latent-space of a StyleGAN, that is disentangled with respect to meaningful image attributes. A stylized numerical study inspired by MR imaging is designed, where the sought-after and the prior image are structurally similar.
  arXiv  Detail & Related papers  (2022-02-17T23:28:10Z)
• Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
  We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms. Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-11-27T12:12:57Z)
• Deep Reparametrization of Multi-Frame Super-Resolution and Denoising [167.42453826365434]
  We propose a deep reparametrization of the maximum a posteriori formulation commonly employed in multi-frame image restoration tasks. Our approach is derived by introducing a learned error metric and a latent representation of the target image. We validate our approach through comprehensive experiments on burst denoising and burst super-resolution datasets.
  arXiv  Detail & Related papers  (2021-08-18T17:57:02Z)
• Regularization via deep generative models: an analysis point of view [8.818465117061205]
  This paper proposes a new way of regularizing an inverse problem in imaging (e.g., deblurring or inpainting) by means of a deep generative neural network. In many cases our technique achieves a clear improvement of the performance and seems to be more robust.
  arXiv  Detail & Related papers  (2021-01-21T15:04:57Z)
• Learning regularization and intensity-gradient-based fidelity for single image super resolution [0.0]
  We study the image degradation progress, and establish degradation model both in intensity and gradient space. A comprehensive data consistency constraint is established for the reconstruction. The proposed fidelity term and designed regularization term are embedded into the regularization framework.
  arXiv  Detail & Related papers  (2020-03-24T07:03:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.