Pose-Guided High-Resolution Appearance Transfer via Progressive Training

• URL: http://arxiv.org/abs/2008.11898v1
• Date: Thu, 27 Aug 2020 03:18:44 GMT
• Title: Pose-Guided High-Resolution Appearance Transfer via Progressive Training
• Authors: Ji Liu, Heshan Liu, Mang-Tik Chiu, Yu-Wing Tai, Chi-Keung Tang
• Abstract summary: We propose a pose-guided appearance transfer network for transferring a given reference appearance to a target pose in unprecedented image resolution. Our network utilizes dense local descriptors including local perceptual loss and local discriminators to refine details. Our model produces high-quality images, which can be further utilized in useful applications such as garment transfer between people.
• Score: 65.92031716146865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel pose-guided appearance transfer network for transferring a given reference appearance to a target pose in unprecedented image resolution (1024 * 1024), given respectively an image of the reference and target person. No 3D model is used. Instead, our network utilizes dense local descriptors including local perceptual loss and local discriminators to refine details, which is trained progressively in a coarse-to-fine manner to produce the high-resolution output to faithfully preserve complex appearance of garment textures and geometry, while hallucinating seamlessly the transferred appearances including those with dis-occlusion. Our progressive encoder-decoder architecture can learn the reference appearance inherent in the input image at multiple scales. Extensive experimental results on the Human3.6M dataset, the DeepFashion dataset, and our dataset collected from YouTube show that our model produces high-quality images, which can be further utilized in useful applications such as garment transfer between people and pose-guided human video generation.

Related papers

• Towards Degradation-Robust Reconstruction in Generalizable NeRF [58.33351079982745]
  Generalizable Radiance Field (GNeRF) across scenes has been proven to be an effective way to avoid per-scene optimization. There has been limited research on the robustness of GNeRFs to different types of degradation present in the source images.
  arXiv  Detail & Related papers  (2024-11-18T16:13:47Z)
• MaRINeR: Enhancing Novel Views by Matching Rendered Images with Nearby References [49.71130133080821]
  MaRINeR is a refinement method that leverages information of a nearby mapping image to improve the rendering of a target viewpoint. We show improved renderings in quantitative metrics and qualitative examples from both explicit and implicit scene representations.
  arXiv  Detail & Related papers  (2024-07-18T17:50:03Z)
• HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos [52.23323966700072]
  We present a framework for acquiring human avatars that are attached with high-resolution physically-based material textures and mesh from monocular video. Our method introduces a novel information fusion strategy to combine the information from the monocular video and synthesize virtual multi-view images. Experiments show that our approach outperforms previous representations in terms of high fidelity, and this explicit result supports deployment on common triangulars.
  arXiv  Detail & Related papers  (2024-05-18T11:49:09Z)
• SARGAN: Spatial Attention-based Residuals for Facial Expression Manipulation [1.7056768055368383]
  We present a novel method named SARGAN that addresses the limitations from three perspectives. We exploited a symmetric encoder-decoder network to attend facial features at multiple scales. Our proposed model performs significantly better than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-30T08:15:18Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)
• NeuralReshaper: Single-image Human-body Retouching with Deep Neural Networks [50.40798258968408]
  We present NeuralReshaper, a novel method for semantic reshaping of human bodies in single images using deep generative networks. Our approach follows a fit-then-reshape pipeline, which first fits a parametric 3D human model to a source human image. To deal with the lack-of-data problem that no paired data exist, we introduce a novel self-supervised strategy to train our network.
  arXiv  Detail & Related papers  (2022-03-20T09:02:13Z)
• HDhuman: High-quality Human Novel-view Rendering from Sparse Views [15.810495442598963]
  We propose HDhuman, which uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network with geometry-guided pixel-wise feature integration. Our approach outperforms all the prior generic or specific methods on both synthetic data and real-world data.
  arXiv  Detail & Related papers  (2022-01-20T13:04:59Z)
• A 3D Mesh-based Lifting-and-Projection Network for Human Pose Transfer [25.681557081096805]
  We propose a lifting-and-projection framework to perform pose transfer in the 3D mesh space. To leverage the human body shape prior, LPNet exploits the topological information of the body mesh. To preserve texture details, ADCNet is introduced to enhance the feature produced by LPNet with the source foreground image.
  arXiv  Detail & Related papers  (2021-09-24T03:03:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.