Image Completion via Inference in Deep Generative Models
- URL: http://arxiv.org/abs/2102.12037v1
- Date: Wed, 24 Feb 2021 02:59:43 GMT
- Title: Image Completion via Inference in Deep Generative Models
- Authors: William Harvey, Saeid Naderiparizi, Frank Wood
- Abstract summary: We consider image completion from the perspective of amortized inference in an image generative model.
We demonstrate superior sample quality and diversity compared to prior art on the CIFAR-10 and FFHQ-256 datasets.
- Score: 16.99337751292915
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We consider image completion from the perspective of amortized inference in
an image generative model. We leverage recent state of the art variational
auto-encoder architectures that have been shown to produce photo-realistic
natural images at non-trivial resolutions. Through amortized inference in such
a model we can train neural artifacts that produce diverse, realistic image
completions even when the vast majority of an image is missing. We demonstrate
superior sample quality and diversity compared to prior art on the CIFAR-10 and
FFHQ-256 datasets. We conclude by describing and demonstrating an application
that requires an in-painting model with the capabilities ours exhibits: the use
of Bayesian optimal experimental design to select the most informative sequence
of small field of view x-rays for chest pathology detection.
Related papers
- Let Real Images be as a Judger, Spotting Fake Images Synthesized with Generative Models [16.900526163168827]
We study the artifact patterns in fake images synthesized by different generative models.
In this paper, we employ natural traces shared only by real images as an additional predictive target in the detector.
Our proposed method gives 96.1% mAP significantly outperforms the baselines.
arXiv Detail & Related papers (2024-03-25T07:58:58Z) - Enhancing Diffusion Models with 3D Perspective Geometry Constraints [10.21800236402905]
We introduce a novel geometric constraint in the training process of generative models to enforce perspective accuracy.
We show that outputs of models trained with this constraint both appear more realistic and improve performance of downstream models trained on generated images.
arXiv Detail & Related papers (2023-12-01T21:56:43Z) - Image Inpainting via Tractable Steering of Diffusion Models [54.13818673257381]
This paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior.
Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs)
We show that our approach can consistently improve the overall quality and semantic coherence of inpainted images with only 10% additional computational overhead.
arXiv Detail & Related papers (2023-11-28T21:14:02Z) - ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with
Diffusion Models [126.35334860896373]
We investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes.
Existing works for higher-resolution generation, such as attention-based and joint-diffusion approaches, cannot well address these issues.
We propose a simple yet effective re-dilation that can dynamically adjust the convolutional perception field during inference.
arXiv Detail & Related papers (2023-10-11T17:52:39Z) - Perceptual Artifacts Localization for Image Synthesis Tasks [59.638307505334076]
We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels.
A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks.
We propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images.
arXiv Detail & Related papers (2023-10-09T10:22:08Z) - GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from
Multi-view Images [79.39247661907397]
We introduce an effective framework Generalizable Model-based Neural Radiance Fields to synthesize free-viewpoint images.
Specifically, we propose a geometry-guided attention mechanism to register the appearance code from multi-view 2D images to a geometry proxy.
arXiv Detail & Related papers (2023-03-24T03:32:02Z) - DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder [73.1010640692609]
We propose a VQ-VAE architecture model with a diffusion decoder (DiVAE) to work as the reconstructing component in image synthesis.
Our model achieves state-of-the-art results and generates more photorealistic images specifically.
arXiv Detail & Related papers (2022-06-01T10:39:12Z) - InvGAN: Invertible GANs [88.58338626299837]
InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.
This allows us to perform image inpainting, merging, and online data augmentation.
arXiv Detail & Related papers (2021-12-08T21:39:00Z) - Perceptual Image Restoration with High-Quality Priori and Degradation
Learning [28.93489249639681]
We show that our model performs well in measuring the similarity between restored and degraded images.
Our simultaneous restoration and enhancement framework generalizes well to real-world complicated degradation types.
arXiv Detail & Related papers (2021-03-04T13:19:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.