3D Human Texture Estimation from a Single Image with Transformers

• URL: http://arxiv.org/abs/2109.02563v1
• Date: Mon, 6 Sep 2021 16:00:20 GMT
• Title: 3D Human Texture Estimation from a Single Image with Transformers
• Authors: Xiangyu Xu, Chen Change Loy
• Abstract summary: We propose a Transformer-based framework for 3D human texture estimation from a single image. We also propose a mask-fusion strategy to combine the advantages of the RGB-based and texture-flow-based models.
• Score: 106.6320286821364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a Transformer-based framework for 3D human texture estimation from a single image. The proposed Transformer is able to effectively exploit the global information of the input image, overcoming the limitations of existing methods that are solely based on convolutional neural networks. In addition, we also propose a mask-fusion strategy to combine the advantages of the RGB-based and texture-flow-based models. We further introduce a part-style loss to help reconstruct high-fidelity colors without introducing unpleasant artifacts. Extensive experiments demonstrate the effectiveness of the proposed method against state-of-the-art 3D human texture estimation approaches both quantitatively and qualitatively.

Related papers

• ConTex-Human: Free-View Rendering of Human from a Single Image with Texture-Consistent Synthesis [49.28239918969784]
  We introduce a texture-consistent back view synthesis module that could transfer the reference image content to the back view. We also propose a visibility-aware patch consistency regularization for texture mapping and refinement combined with the synthesized back view texture.
  arXiv  Detail & Related papers  (2023-11-28T13:55:53Z)
• PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation [89.09455618184239]
  Recent advances in text-to-3D human generation have been groundbreaking. We propose a model called PaintHuman to address the challenges from two aspects. We use the depth map as a guidance to ensure realistic semantically aligned textures.
  arXiv  Detail & Related papers  (2023-10-14T00:37:16Z)
• Refining 3D Human Texture Estimation from a Single Image [3.8761064607384195]
  Estimating 3D human texture from a single image is essential in graphics and vision. We propose a framework that adaptively samples the input by a deformable convolution where offsets are learned via a deep neural network.
  arXiv  Detail & Related papers  (2023-03-06T19:53:50Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing [41.34640834483265]
  We present PHORHUM, a novel, end-to-end trainable, deep neural network methodology for photorealistic 3D human reconstruction given just a monocular RGB image. Our pixel-aligned method estimates detailed 3D geometry and, for the first time, the unshaded surface color together with the scene illumination.
  arXiv  Detail & Related papers  (2022-04-19T14:06:16Z)
• Differentiable Rendering for Synthetic Aperture Radar Imagery [0.0]
  We propose an approach for differentiable rendering of Synthetic Aperture Radar (SAR) imagery, which combines methods from 3D computer graphics with neural rendering. We demonstrate the approach on the inverse graphics problem of 3D Object Reconstruction from limited SAR imagery using high-fidelity simulated SAR data.
  arXiv  Detail & Related papers  (2022-04-04T05:27:40Z)
• NeuralReshaper: Single-image Human-body Retouching with Deep Neural Networks [50.40798258968408]
  We present NeuralReshaper, a novel method for semantic reshaping of human bodies in single images using deep generative networks. Our approach follows a fit-then-reshape pipeline, which first fits a parametric 3D human model to a source human image. To deal with the lack-of-data problem that no paired data exist, we introduce a novel self-supervised strategy to train our network.
  arXiv  Detail & Related papers  (2022-03-20T09:02:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.