Controllable Person Image Synthesis with Attribute-Decomposed GAN

• URL: http://arxiv.org/abs/2003.12267v4
• Date: Sun, 19 Jul 2020 05:32:31 GMT
• Title: Controllable Person Image Synthesis with Attribute-Decomposed GAN
• Authors: Yifang Men, Yiming Mao, Yuning Jiang, Wei-Ying Ma, Zhouhui Lian
• Abstract summary: This paper introduces the Attribute-Decomposed GAN, a novel generative model for controllable person image synthesis. The core idea of the proposed model is to embed human attributes into the latent space as independent codes. Experimental results demonstrate the proposed method's superiority over the state of the art in pose transfer.
• Score: 27.313729413684012
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces the Attribute-Decomposed GAN, a novel generative model for controllable person image synthesis, which can produce realistic person images with desired human attributes (e.g., pose, head, upper clothes and pants) provided in various source inputs. The core idea of the proposed model is to embed human attributes into the latent space as independent codes and thus achieve flexible and continuous control of attributes via mixing and interpolation operations in explicit style representations. Specifically, a new architecture consisting of two encoding pathways with style block connections is proposed to decompose the original hard mapping into multiple more accessible subtasks. In source pathway, we further extract component layouts with an off-the-shelf human parser and feed them into a shared global texture encoder for decomposed latent codes. This strategy allows for the synthesis of more realistic output images and automatic separation of un-annotated attributes. Experimental results demonstrate the proposed method's superiority over the state of the art in pose transfer and its effectiveness in the brand-new task of component attribute transfer.

Related papers

• Style-Extracting Diffusion Models for Semi-Supervised Histopathology Segmentation [6.479933058008389]
  Style-Extracting Diffusion Models generate images with unseen characteristics beneficial for downstream tasks. In this work, we show the capability of our method on a natural image dataset as a proof-of-concept. We verify the added value of the generated images by showing improved segmentation results and lower performance variability between patients.
  arXiv  Detail & Related papers  (2024-03-21T14:36:59Z)
• Disentangled Representation Learning for Controllable Person Image Generation [29.719070087384512]
  We propose a novel framework named DRL-CPG to learn disentangled latent representation for controllable person image generation. To our knowledge, we are the first to learn disentangled latent representations with transformers for person image generation.
  arXiv  Detail & Related papers  (2023-12-10T07:15:58Z)
• Controllable Multi-domain Semantic Artwork Synthesis [17.536225601718687]
  We propose a dataset that contains 40,000 images of artwork from 4 different domains with their corresponding semantic label maps. We generate the dataset by first extracting semantic maps from landscape photography. We then propose a conditional Generative Adrial Network (GAN)-based approach to generate high-quality artwork.
  arXiv  Detail & Related papers  (2023-08-19T21:16:28Z)
• Composer: Creative and Controllable Image Synthesis with Composable Conditions [57.78533372393828]
  Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability. This work offers a new generation paradigm that allows flexible control of the output image, such as spatial layout and palette, while maintaining the synthesis quality and model creativity.
  arXiv  Detail & Related papers  (2023-02-20T05:48:41Z)
• High-Fidelity Guided Image Synthesis with Latent Diffusion Models [50.39294302741698]
  The proposed approach outperforms the previous state-of-the-art by over 85.32% on the overall user satisfaction scores. Human user study results show that the proposed approach outperforms the previous state-of-the-art by over 85.32% on the overall user satisfaction scores.
  arXiv  Detail & Related papers  (2022-11-30T15:43:20Z)
• Person Image Synthesis via Denoising Diffusion Model [116.34633988927429]
  We show how denoising diffusion models can be applied for high-fidelity person image synthesis. Our results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios.
  arXiv  Detail & Related papers  (2022-11-22T18:59:50Z)
• Dynamic Prototype Mask for Occluded Person Re-Identification [88.7782299372656]
  Existing methods mainly address this issue by employing body clues provided by an extra network to distinguish the visible part. We propose a novel Dynamic Prototype Mask (DPM) based on two self-evident prior knowledge. Under this condition, the occluded representation could be well aligned in a selected subspace spontaneously.
  arXiv  Detail & Related papers  (2022-07-19T03:31:13Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Style [5.912209564607099]
  We propose a method for attribute controlled image synthesis from layout. We extend a state-of-the-art approach for layout-to-image generation to condition individual objects on attributes. Our results show that our method can successfully control the fine-grained details of individual objects when modelling complex scenes with multiple objects.
  arXiv  Detail & Related papers  (2021-03-25T10:09:45Z)
• Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision [83.33283892171562]
  Example-guided image synthesis has recently been attempted to synthesize an image from a semantic label map and an exemplary image. In this paper, we tackle a more challenging and general task, where the exemplar is an arbitrary scene image that is semantically different from the given label map. We propose an end-to-end network for joint global and local feature alignment and synthesis.
  arXiv  Detail & Related papers  (2020-04-18T18:17:40Z)
• Continuous learning of face attribute synthesis [4.786157629600151]
  The generative adversarial network (GAN) exhibits great superiority in the face attribute synthesis task. Existing methods have very limited effects on the expansion of new attributes. A continuous learning method for face attribute synthesis is proposed in this work.
  arXiv  Detail & Related papers  (2020-04-15T06:44:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.