Correspondence Learning for Controllable Person Image Generation
- URL: http://arxiv.org/abs/2012.12440v1
- Date: Wed, 23 Dec 2020 01:35:00 GMT
- Title: Correspondence Learning for Controllable Person Image Generation
- Authors: Shilong Shen
- Abstract summary: We present a generative model for controllable person image synthesis, $i.e.$, which can be applied to pose-guided person image synthesis.
By explicitly establishing the dense correspondence between the target pose and the source image, we can effectively address the misalignment introduced by pose tranfer.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present a generative model for controllable person image synthesis,as
shown in Figure , which can be applied to pose-guided person image synthesis,
$i.e.$, converting the pose of a source person image to the target pose while
preserving the texture of that source person image, and clothing-guided person
image synthesis, $i.e.$, changing the clothing texture of a source person image
to the desired clothing texture. By explicitly establishing the dense
correspondence between the target pose and the source image, we can effectively
address the misalignment introduced by pose tranfer and generate high-quality
images. Specifically, we first generate the target semantic map under the
guidence of the target pose, which can provide more accurate pose
representation and structural constraints during the generation process. Then,
decomposed attribute encoder is used to extract the component features, which
not only helps to establish a more accurate dense correspondence, but also
realizes the clothing-guided person generation. After that, we will establish a
dense correspondence between the target pose and the source image within the
sharded domain. The source image feature is warped according to the dense
correspondence to flexibly account for deformations. Finally, the network
renders image based on the warped source image feature and the target pose.
Experimental results show that our method is superior to state-of-the-art
methods in pose-guided person generation and its effectiveness in
clothing-guided person generation.
Related papers
- HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for
Controllable Text-Driven Person Image Generation [73.3790833537313]
Controllable person image generation promotes a wide range of applications such as digital human interaction and virtual try-on.
We propose HumanDiffusion, a coarse-to-fine alignment diffusion framework, for text-driven person image generation.
arXiv Detail & Related papers (2022-11-11T14:30:34Z) - Pose Guided Human Image Synthesis with Partially Decoupled GAN [25.800174118151638]
Pose Guided Human Image Synthesis (PGHIS) is a challenging task of transforming a human image from the reference pose to a target pose.
We propose a method by decoupling the human body into several parts to guide the synthesis of a realistic image of the person.
In addition, we design a multi-head attention-based module for PGHIS.
arXiv Detail & Related papers (2022-10-07T15:31:37Z) - Controllable Person Image Synthesis with Spatially-Adaptive Warped
Normalization [72.65828901909708]
Controllable person image generation aims to produce realistic human images with desirable attributes.
We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters.
We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
arXiv Detail & Related papers (2021-05-31T07:07:44Z) - HumanGAN: A Generative Model of Humans Images [78.6284090004218]
We present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style.
Our model encodes part-based latent appearance vectors in a normalized pose-independent space and warps them to different poses, it preserves body and clothing appearance under varying posture.
arXiv Detail & Related papers (2021-03-11T19:00:38Z) - PISE: Person Image Synthesis and Editing with Decoupled GAN [64.70360318367943]
We propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing.
For human pose transfer, we first synthesize a human parsing map aligned with the target pose to represent the shape of clothing.
To decouple the shape and style of clothing, we propose joint global and local per-region encoding and normalization.
arXiv Detail & Related papers (2021-03-06T04:32:06Z) - Style and Pose Control for Image Synthesis of Humans from a Single
Monocular View [78.6284090004218]
StylePoseGAN is a non-controllable generator to accept conditioning of pose and appearance separately.
Our network can be trained in a fully supervised way with human images to disentangle pose, appearance and body parts.
StylePoseGAN achieves state-of-the-art image generation fidelity on common perceptual metrics.
arXiv Detail & Related papers (2021-02-22T18:50:47Z) - Structure-aware Person Image Generation with Pose Decomposition and
Semantic Correlation [29.727033198797518]
We propose a structure-aware flow based method for high-quality person image generation.
We decompose the human body into different semantic parts and apply different networks to predict the flow fields for these parts separately.
Our method can generate high-quality results under large pose discrepancy and outperforms state-of-the-art methods in both qualitative and quantitative comparisons.
arXiv Detail & Related papers (2021-02-05T03:07:57Z) - Person image generation with semantic attention network for person
re-identification [9.30413920076019]
We propose a novel person pose-guided image generation method, which is called the semantic attention network.
The network consists of several semantic attention blocks, where each block attends to preserve and update the pose code and the clothing textures.
Compared with other methods, our network can characterize better body shape and keep clothing attributes, simultaneously.
arXiv Detail & Related papers (2020-08-18T12:18:51Z) - Generating Person Images with Appearance-aware Pose Stylizer [66.44220388377596]
We present a novel end-to-end framework to generate realistic person images based on given person poses and appearances.
The core of our framework is a novel generator called Appearance-aware Pose Stylizer (APS) which generates human images by coupling the target pose with the conditioned person appearance progressively.
arXiv Detail & Related papers (2020-07-17T15:58:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.