Image Restoration from Parametric Transformations using Generative
Models
- URL: http://arxiv.org/abs/2005.14036v2
- Date: Tue, 16 Jun 2020 12:09:41 GMT
- Title: Image Restoration from Parametric Transformations using Generative
Models
- Authors: Kalliopi Basioti, George V. Moustakides
- Abstract summary: We develop optimum techniques for various image restoration problems using generative models.
Our approach is capable of restoring images that are distorted by transformations even when the latter contain unknown parameters.
We extend our method to accommodate mixtures of multiple images where each image is described by its own generative model.
- Score: 4.467248776406006
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: When images are statistically described by a generative model we can use this
information to develop optimum techniques for various image restoration
problems as inpainting, super-resolution, image coloring, generative model
inversion, etc. With the help of the generative model it is possible to
formulate, in a natural way, these restoration problems as Statistical
estimation problems. Our approach, by combining maximum a-posteriori
probability with maximum likelihood estimation, is capable of restoring images
that are distorted by transformations even when the latter contain unknown
parameters. The resulting optimization is completely defined with no parameters
requiring tuning. This must be compared with the current state of the art which
requires exact knowledge of the transformations and contains regularizer terms
with weights that must be properly defined. Finally, we must mention that we
extend our method to accommodate mixtures of multiple images where each image
is described by its own generative model and we are able of successfully
separating each participating image from a single mixture.
Related papers
- Imaging Signal Recovery Using Neural Network Priors Under Uncertain Forward Model Parameters [0.7724713939814069]
Inverse imaging problems (IIPs) arise in various applications, with the main objective of reconstructing an image from its compressed measurements.
We propose a novel Moment-Aggregation (MA) framework that is compatible with the popular IIP solution by using a neural network prior.
We theoretically demonstrate the convergence of the MA framework, which has a similar complexity with reconstruction under the known forward model parameters.
arXiv Detail & Related papers (2024-05-05T14:12:48Z) - Image Inpainting via Tractable Steering of Diffusion Models [54.13818673257381]
This paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior.
Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs)
We show that our approach can consistently improve the overall quality and semantic coherence of inpainted images with only 10% additional computational overhead.
arXiv Detail & Related papers (2023-11-28T21:14:02Z) - DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion [144.9653045465908]
We propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM)
Our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
arXiv Detail & Related papers (2023-03-13T04:06:42Z) - Invertible Rescaling Network and Its Extensions [118.72015270085535]
In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective.
We develop invertible models to generate valid degraded images and transform the distribution of lost contents.
Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
arXiv Detail & Related papers (2022-10-09T06:58:58Z) - One Size Fits All: Hypernetwork for Tunable Image Restoration [5.33024001730262]
We introduce a novel approach for tunable image restoration that achieves the accuracy of multiple models, each optimized for a different level of degradation.
Our model can be optimized to restore as many degradation levels as required with a constant number of parameters and for various image restoration tasks.
arXiv Detail & Related papers (2022-06-13T08:33:14Z) - Image Generation with Multimodal Priors using Denoising Diffusion
Probabilistic Models [54.1843419649895]
A major challenge in using generative models to accomplish this task is the lack of paired data containing all modalities and corresponding outputs.
We propose a solution based on a denoising diffusion probabilistic synthesis models to generate images under multi-model priors.
arXiv Detail & Related papers (2022-06-10T12:23:05Z) - Paired Image-to-Image Translation Quality Assessment Using Multi-Method
Fusion [0.0]
This paper proposes a novel approach that combines signals of image quality between paired source and transformation to predict the latter's similarity with a hypothetical ground truth.
We trained a Multi-Method Fusion (MMF) model via an ensemble of gradient-boosted regressors to predict Deep Image Structure and Texture Similarity (DISTS)
Analysis revealed the task to be feature-constrained, introducing a trade-off at inference between metric time and prediction accuracy.
arXiv Detail & Related papers (2022-05-09T11:05:15Z) - FaceCook: Face Generation Based on Linear Scaling Factors [11.682904465909003]
We propose a new approach to mapping the latent vectors of the generative model to the scaling factors.
The proposed method outperforms the baseline in terms of image diversity.
arXiv Detail & Related papers (2021-09-08T08:31:40Z) - Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
Blind image restoration is a common yet challenging problem in computer vision.
We propose a novel blind image restoration method, aiming to integrate both the advantages of them.
Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
arXiv Detail & Related papers (2020-08-25T03:30:53Z) - PaMIR: Parametric Model-Conditioned Implicit Representation for
Image-based Human Reconstruction [67.08350202974434]
We propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function.
We show that our method achieves state-of-the-art performance for image-based 3D human reconstruction in the cases of challenging poses and clothing types.
arXiv Detail & Related papers (2020-07-08T02:26:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.