TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation

• URL: http://arxiv.org/abs/2403.12906v1
• Date: Tue, 19 Mar 2024 17:02:07 GMT
• Title: TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation
• Authors: Yufei Liu, Junwei Zhu, Junshu Tang, Shijie Zhang, Jiangning Zhang, Weijian Cao, Chengjie Wang, Yunsheng Wu, Dongjin Huang,
• Abstract summary: TexDreamer is the first zero-shot multimodal high-fidelity 3D human texture generation model. We introduce ArTicuLated humAn textureS, the largest high-resolution (1024 X 1024) 3D human texture dataset.
• Score: 41.959089177835764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Texturing 3D humans with semantic UV maps remains a challenge due to the difficulty of acquiring reasonably unfolded UV. Despite recent text-to-3D advancements in supervising multi-view renderings using large text-to-image (T2I) models, issues persist with generation speed, text consistency, and texture quality, resulting in data scarcity among existing datasets. We present TexDreamer, the first zero-shot multimodal high-fidelity 3D human texture generation model. Utilizing an efficient texture adaptation finetuning strategy, we adapt large T2I model to a semantic UV structure while preserving its original generalization capability. Leveraging a novel feature translator module, the trained model is capable of generating high-fidelity 3D human textures from either text or image within seconds. Furthermore, we introduce ArTicuLated humAn textureS (ATLAS), the largest high-resolution (1024 X 1024) 3D human texture dataset which contains 50k high-fidelity textures with text descriptions.

Related papers

• TEXGen: a Generative Diffusion Model for Mesh Textures [63.43159148394021]
  We focus on the fundamental problem of learning in the UV texture space itself. We propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. We train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images.
  arXiv  Detail & Related papers  (2024-11-22T05:22:11Z)
• FAMOUS: High-Fidelity Monocular 3D Human Digitization Using View Synthesis [51.193297565630886]
  The challenge of accurately inferring texture remains, particularly in obscured areas such as the back of a person in frontal-view images. This limitation in texture prediction largely stems from the scarcity of large-scale and diverse 3D datasets. We propose leveraging extensive 2D fashion datasets to enhance both texture and shape prediction in 3D human digitization.
  arXiv  Detail & Related papers  (2024-10-13T01:25:05Z)
• Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objects [54.80813150893719]
  We introduce Meta 3D TextureGen: a new feedforward method comprised of two sequential networks aimed at generating high-quality textures in less than 20 seconds. Our method state-of-the-art results in quality and speed by conditioning a text-to-image model on 3D semantics in 2D space and fusing them into a complete and high-resolution UV texture map. In addition, we introduce a texture enhancement network that is capable of up-scaling any texture by an arbitrary ratio, producing 4k pixel resolution textures.
  arXiv  Detail & Related papers  (2024-07-02T17:04:34Z)
• EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion [5.158983929861116]
  We present EucliDreamer, a simple and effective method to generate textures for 3D models given text and prompts. The texture is parametized as an implicit function on the 3D surface, which is optimized with the Score Distillation Sampling (SDS) process and differentiable rendering.
  arXiv  Detail & Related papers  (2024-04-16T04:44:16Z)
• TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models [77.85129451435704]
  We present a new method to synthesize textures for 3D, using large-scale-guided image diffusion models. Specifically, we leverage latent diffusion models, apply the set denoising model and aggregate denoising text map.
  arXiv  Detail & Related papers  (2023-10-20T19:15:29Z)
• AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis [78.17671694498185]
  We propose AUV-Net which learns to embed 3D surfaces into a 2D aligned UV space. As a result, textures are aligned across objects, and can thus be easily synthesized by generative models of images. The learned UV mapping and aligned texture representations enable a variety of applications including texture transfer, texture synthesis, and textured single view 3D reconstruction.
  arXiv  Detail & Related papers  (2022-04-06T21:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.