Hierarchy Composition GAN for High-fidelity Image Synthesis

• URL: http://arxiv.org/abs/1905.04693v5
• Date: Wed, 19 Apr 2023 20:58:17 GMT
• Title: Hierarchy Composition GAN for High-fidelity Image Synthesis
• Authors: Fangneng Zhan, Jiaxing Huang and Shijian Lu
• Abstract summary: This paper presents an innovative Hierarchical Composition GAN (HIC-GAN) HIC-GAN incorporates image synthesis in geometry and appearance domains into an end-to-end trainable network. Experiments on scene text image synthesis, portrait editing and indoor rendering tasks show that the proposed HIC-GAN achieves superior synthesis performance qualitatively and quantitatively.
• Score: 57.32311953820988
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the rapid progress of generative adversarial networks (GANs) in image synthesis in recent years, the existing image synthesis approaches work in either geometry domain or appearance domain alone which often introduces various synthesis artifacts. This paper presents an innovative Hierarchical Composition GAN (HIC-GAN) that incorporates image synthesis in geometry and appearance domains into an end-to-end trainable network and achieves superior synthesis realism in both domains simultaneously. We design an innovative hierarchical composition mechanism that is capable of learning realistic composition geometry and handling occlusions while multiple foreground objects are involved in image composition. In addition, we introduce a novel attention mask mechanism that guides to adapt the appearance of foreground objects which also helps to provide better training reference for learning in geometry domain. Extensive experiments on scene text image synthesis, portrait editing and indoor rendering tasks show that the proposed HIC-GAN achieves superior synthesis performance qualitatively and quantitatively.

Related papers

• SAMPLING: Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image [60.52991173059486]
  We introduce SAMPLING, a Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image. Our method demonstrates considerable performance gains in large-scale unbounded outdoor scenes using a single image on the KITTI dataset.
  arXiv  Detail & Related papers  (2023-09-12T15:33:09Z)
• GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images [79.39247661907397]
  We introduce an effective framework Generalizable Model-based Neural Radiance Fields to synthesize free-viewpoint images. Specifically, we propose a geometry-guided attention mechanism to register the appearance code from multi-view 2D images to a geometry proxy.
  arXiv  Detail & Related papers  (2023-03-24T03:32:02Z)
• A geometrically aware auto-encoder for multi-texture synthesis [1.2891210250935146]
  We propose an auto-encoder architecture for multi-texture synthesis. Images are embedded in a compact and geometrically consistent latent space. Texture synthesis and tasks can be performed directly from these latent codes.
  arXiv  Detail & Related papers  (2023-02-03T09:28:39Z)
• SIAN: Style-Guided Instance-Adaptive Normalization for Multi-Organ Histopathology Image Synthesis [63.845552349914186]
  We propose a style-guided instance-adaptive normalization (SIAN) to synthesize realistic color distributions and textures for different organs. The four phases work together and are integrated into a generative network to embed image semantics, style, and instance-level boundaries.
  arXiv  Detail & Related papers  (2022-09-02T16:45:46Z)
• Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning [40.544844623958426]
  We propose a novel Semantic-Driven Generative Adversarial Network to address the above issues. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator. We construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the IntrA-class Semantic Graph (IASG) and the InteR-class Structure Graph (IRSG)
  arXiv  Detail & Related papers  (2022-01-05T13:14:14Z)
• A Survey on Adversarial Image Synthesis [0.0]
  We provide a taxonomy of methods used in image synthesis, review different models for text-to-image synthesis and image-to-image translation, and discuss some evaluation metrics as well as possible future research directions in image synthesis with GAN.
  arXiv  Detail & Related papers  (2021-06-30T13:31:48Z)
• Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition [61.87842307164351]
  We first propose an Identity-Aware CycleGAN (IACycleGAN) model that applies a new perceptual loss to supervise the image generation network. It improves CycleGAN on photo-sketch synthesis by paying more attention to the synthesis of key facial regions, such as eyes and nose. We develop a mutual optimization procedure between the synthesis model and the recognition model, which iteratively synthesizes better images by IACycleGAN.
  arXiv  Detail & Related papers  (2021-03-30T01:30:08Z)
• Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis [16.097324852253912]
  We introduce a novel augmentation scheme designed specifically for generative adversarial networks (GANs) We propose to randomly warp object shapes in the semantic label maps used as an input to the generator. The local shape discrepancies between the warped and non-warped label maps and images enable the GAN to learn better the structural and geometric details of the scene.
  arXiv  Detail & Related papers  (2020-11-25T10:55:26Z)
• Adversarial Image Composition with Auxiliary Illumination [53.89445873577062]
  We propose an Adversarial Image Composition Net (AIC-Net) that achieves realistic image composition. A novel branched generation mechanism is proposed, which disentangles the generation of shadows and the transfer of foreground styles. Experiments on pedestrian and car composition tasks show that the proposed AIC-Net achieves superior composition performance.
  arXiv  Detail & Related papers  (2020-09-17T12:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.