PISE: Person Image Synthesis and Editing with Decoupled GAN

• URL: http://arxiv.org/abs/2103.04023v1
• Date: Sat, 6 Mar 2021 04:32:06 GMT
• Title: PISE: Person Image Synthesis and Editing with Decoupled GAN
• Authors: Jinsong Zhang, Kun Li, Yu-Kun Lai, Jingyu Yang
• Abstract summary: We propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing. For human pose transfer, we first synthesize a human parsing map aligned with the target pose to represent the shape of clothing. To decouple the shape and style of clothing, we propose joint global and local per-region encoding and normalization.
• Score: 64.70360318367943
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Person image synthesis, e.g., pose transfer, is a challenging problem due to large variation and occlusion. Existing methods have difficulties predicting reasonable invisible regions and fail to decouple the shape and style of clothing, which limits their applications on person image editing. In this paper, we propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing, which is able to generate realistic person images with desired poses, textures, or semantic layouts. For human pose transfer, we first synthesize a human parsing map aligned with the target pose to represent the shape of clothing by a parsing generator, and then generate the final image by an image generator. To decouple the shape and style of clothing, we propose joint global and local per-region encoding and normalization to predict the reasonable style of clothing for invisible regions. We also propose spatial-aware normalization to retain the spatial context relationship in the source image. The results of qualitative and quantitative experiments demonstrate the superiority of our model on human pose transfer. Besides, the results of texture transfer and region editing show that our model can be applied to person image editing.

Related papers

• A Two-stage Personalized Virtual Try-on Framework with Shape Control and Texture Guidance [7.302929117437442]
  This paper proposes a brand new personalized virtual try-on model (PE-VITON), which uses the two stages (shape control and texture guidance) to decouple the clothing attributes. The proposed model can effectively solve the problems of weak reduction of clothing folds, poor generation effect under complex human posture, blurred edges of clothing, and unclear texture styles in traditional try-on methods.
  arXiv  Detail & Related papers  (2023-12-24T13:32:55Z)
• Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN [88.62422914645066]
  We present an algorithm for re-rendering a person from a single image under arbitrary poses. Existing methods often have difficulties in hallucinating occluded contents photo-realistically while preserving the identity and fine details in the source image. We show that our method compares favorably against the state-of-the-art algorithms in both quantitative evaluation and visual comparison.
  arXiv  Detail & Related papers  (2021-09-13T17:59:33Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• HumanGAN: A Generative Model of Humans Images [78.6284090004218]
  We present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style. Our model encodes part-based latent appearance vectors in a normalized pose-independent space and warps them to different poses, it preserves body and clothing appearance under varying posture.
  arXiv  Detail & Related papers  (2021-03-11T19:00:38Z)
• Style and Pose Control for Image Synthesis of Humans from a Single Monocular View [78.6284090004218]
  StylePoseGAN is a non-controllable generator to accept conditioning of pose and appearance separately. Our network can be trained in a fully supervised way with human images to disentangle pose, appearance and body parts. StylePoseGAN achieves state-of-the-art image generation fidelity on common perceptual metrics.
  arXiv  Detail & Related papers  (2021-02-22T18:50:47Z)
• Correspondence Learning for Controllable Person Image Generation [0.0]
  We present a generative model for controllable person image synthesis, $i.e.$, which can be applied to pose-guided person image synthesis. By explicitly establishing the dense correspondence between the target pose and the source image, we can effectively address the misalignment introduced by pose tranfer.
  arXiv  Detail & Related papers  (2020-12-23T01:35:00Z)
• Pose-Guided Human Animation from a Single Image in the Wild [83.86903892201656]
  We present a new pose transfer method for synthesizing a human animation from a single image of a person controlled by a sequence of body poses. Existing pose transfer methods exhibit significant visual artifacts when applying to a novel scene. We design a compositional neural network that predicts the silhouette, garment labels, and textures. We are able to synthesize human animations that can preserve the identity and appearance of the person in a temporally coherent way without any fine-tuning of the network on the testing scene.
  arXiv  Detail & Related papers  (2020-12-07T15:38:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.