Nested Scale Editing for Conditional Image Synthesis

• URL: http://arxiv.org/abs/2006.02038v1
• Date: Wed, 3 Jun 2020 04:29:21 GMT
• Title: Nested Scale Editing for Conditional Image Synthesis
• Authors: Lingzhi Zhang, Jiancong Wang, Yinshuang Xu, Jie Min, Tarmily Wen, James C. Gee, Jianbo Shi
• Abstract summary: We propose an image synthesis approach that provides stratified navigation in the latent code space. With a tiny amount of partial or very low-resolution image, our approach can consistently out-perform state-of-the-art counterparts.
• Score: 19.245119912119947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose an image synthesis approach that provides stratified navigation in the latent code space. With a tiny amount of partial or very low-resolution image, our approach can consistently out-perform state-of-the-art counterparts in terms of generating the closest sampled image to the ground truth. We achieve this through scale-independent editing while expanding scale-specific diversity. Scale-independence is achieved with a nested scale disentanglement loss. Scale-specific diversity is created by incorporating a progressive diversification constraint. We introduce semantic persistency across the scales by sharing common latent codes. Together they provide better control of the image synthesis process. We evaluate the effectiveness of our proposed approach through various tasks, including image outpainting, image superresolution, and cross-domain image translation.

Related papers

• Generative Powers of Ten [60.6740997942711]
  We present a method that uses a text-to-image model to generate consistent content across multiple image scales. We achieve this through a joint multi-scale diffusion sampling approach. Our method enables deeper levels of zoom than traditional super-resolution methods.
  arXiv  Detail & Related papers  (2023-12-04T18:59:25Z)
• Layered Rendering Diffusion Model for Zero-Shot Guided Image Synthesis [60.260724486834164]
  This paper introduces innovative solutions to enhance spatial controllability in diffusion models reliant on text queries. We present two key innovations: Vision Guidance and the Layered Rendering Diffusion framework. We apply our method to three practical applications: bounding box-to-image, semantic mask-to-image and image editing.
  arXiv  Detail & Related papers  (2023-11-30T10:36:19Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Arbitrary-Scale Image Synthesis [149.0290830305808]
  Positional encodings have enabled recent works to train a single adversarial network that can generate images of different scales. We propose the design of scale-consistent positional encodings invariant to our generator's transformations layers. We show competitive results for a continuum of scales on various commonly used datasets for image synthesis.
  arXiv  Detail & Related papers  (2022-04-05T15:10:43Z)
• Diverse Semantic Image Synthesis via Probability Distribution Modeling [103.88931623488088]
  We propose a novel diverse semantic image synthesis framework. Our method can achieve superior diversity and comparable quality compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-11T18:59:25Z)
• Panoptic-based Image Synthesis [32.82903428124024]
  Conditional image synthesis serves various applications for content editing to content generation. We propose a panoptic aware image synthesis network to generate high fidelity and photorealistic images conditioned on panoptic maps.
  arXiv  Detail & Related papers  (2020-04-21T20:40:53Z)
• Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimation [54.17177006826262]
  We develop a new generic conditional image synthesis method based on Implicit Maximum Likelihood Estimation (IMLE) We demonstrate improved multimodal image synthesis performance on two tasks, single image super-resolution and image synthesis from scene layouts.
  arXiv  Detail & Related papers  (2020-04-07T03:06:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.