Prior Image-Constrained Reconstruction using Style-Based Generative
Models
- URL: http://arxiv.org/abs/2102.12525v2
- Date: Mon, 14 Jun 2021 23:22:13 GMT
- Title: Prior Image-Constrained Reconstruction using Style-Based Generative
Models
- Authors: Varun A. Kelkar, Mark A. Anastasio
- Abstract summary: We propose a framework for estimating an object of interest that is semantically related to a known prior image.
An optimization problem is formulated in the disentangled latent space of a style-based generative model.
Semantically meaningful constraints are imposed using the disentangled latent representation of the prior image.
- Score: 15.757204774959366
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Obtaining a useful estimate of an object from highly incomplete imaging
measurements remains a holy grail of imaging science. Deep learning methods
have shown promise in learning object priors or constraints to improve the
conditioning of an ill-posed imaging inverse problem. In this study, a
framework for estimating an object of interest that is semantically related to
a known prior image, is proposed. An optimization problem is formulated in the
disentangled latent space of a style-based generative model, and semantically
meaningful constraints are imposed using the disentangled latent representation
of the prior image. Stable recovery from incomplete measurements with the help
of a prior image is theoretically analyzed. Numerical experiments demonstrating
the superior performance of our approach as compared to related methods are
presented.
Related papers
- Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation [50.16004183320537]
We describe a method for recovering the irradiance underlying a collection of images corrupted by atmospheric turbulence.
We select one of the images as a reference, and model the deformation in this image by the aggregation of the optical flow from it to other images.
We achieve state-of-the-art performance despite its simplicity.
arXiv Detail & Related papers (2024-05-06T17:39:53Z) - Disentangled Pre-training for Image Matting [74.10407744483526]
Image matting requires high-quality pixel-level human annotations to support the training of a deep model.
We propose a self-supervised pre-training approach that can leverage infinite numbers of data to boost the matting performance.
arXiv Detail & Related papers (2023-04-03T08:16:02Z) - TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose
Estimation [55.94900327396771]
We introduce neural texture learning for 6D object pose estimation from synthetic data.
We learn to predict realistic texture of objects from real image collections.
We learn pose estimation from pixel-perfect synthetic data.
arXiv Detail & Related papers (2022-12-25T13:36:32Z) - Deformable Surface Reconstruction via Riemannian Metric Preservation [9.74575494970697]
Estimating the pose of an object from a monocular image is an inverse problem fundamental in computer vision.
This paper presents an approach to inferring continuous deformable surfaces from a sequence of images.
arXiv Detail & Related papers (2022-12-22T10:45:08Z) - Bayesian Inversion for Nonlinear Imaging Models using Deep Generative
Priors [24.544313203472992]
We develop a tractable posterior-sampling scheme based on the Metropolis-adjusted Langevin algorithm for the class of nonlinear inverse problems.
We illustrate the advantages of our framework by applying it to two nonlinear imaging modalities-phase retrieval and optical diffraction tomography.
arXiv Detail & Related papers (2022-03-18T17:47:29Z) - Object-centric and memory-guided normality reconstruction for video
anomaly detection [56.64792194894702]
This paper addresses anomaly detection problem for videosurveillance.
Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy.
Our model learns object-centric normal patterns without seeing anomalous samples during training.
arXiv Detail & Related papers (2022-03-07T19:28:39Z) - Prior image-based medical image reconstruction using a style-based
generative adversarial network [15.757204774959366]
This work proposes to use a style-based generative adversarial network (StyleGAN) to constrain an image reconstruction problem.
An optimization problem is formulated in the intermediate latent-space of a StyleGAN, that is disentangled with respect to meaningful image attributes.
A stylized numerical study inspired by MR imaging is designed, where the sought-after and the prior image are structurally similar.
arXiv Detail & Related papers (2022-02-17T23:28:10Z) - Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms.
Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
arXiv Detail & Related papers (2021-11-27T12:12:57Z) - Deep Reparametrization of Multi-Frame Super-Resolution and Denoising [167.42453826365434]
We propose a deep reparametrization of the maximum a posteriori formulation commonly employed in multi-frame image restoration tasks.
Our approach is derived by introducing a learned error metric and a latent representation of the target image.
We validate our approach through comprehensive experiments on burst denoising and burst super-resolution datasets.
arXiv Detail & Related papers (2021-08-18T17:57:02Z) - Regularization via deep generative models: an analysis point of view [8.818465117061205]
This paper proposes a new way of regularizing an inverse problem in imaging (e.g., deblurring or inpainting) by means of a deep generative neural network.
In many cases our technique achieves a clear improvement of the performance and seems to be more robust.
arXiv Detail & Related papers (2021-01-21T15:04:57Z) - Learning regularization and intensity-gradient-based fidelity for single
image super resolution [0.0]
We study the image degradation progress, and establish degradation model both in intensity and gradient space.
A comprehensive data consistency constraint is established for the reconstruction.
The proposed fidelity term and designed regularization term are embedded into the regularization framework.
arXiv Detail & Related papers (2020-03-24T07:03:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.