UVMap-ID: A Controllable and Personalized UV Map Generative Model

• URL: http://arxiv.org/abs/2404.14568v2
• Date: Fri, 9 Aug 2024 09:05:09 GMT
• Title: UVMap-ID: A Controllable and Personalized UV Map Generative Model
• Authors: Weijie Wang, Jichao Zhang, Chang Liu, Xia Li, Xingqian Xu, Humphrey Shi, Nicu Sebe, Bruno Lepri,
• Abstract summary: We introduce UVMap-ID, a controllable and personalized UV Map generative model. Unlike traditional large-scale training methods in 2D, we propose to fine-tune a pre-trained text-to-image diffusion model. Both quantitative and qualitative analyses demonstrate the effectiveness of our method in controllable and personalized UV Map generation.
• Score: 67.71022515856653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, diffusion models have made significant strides in synthesizing realistic 2D human images based on provided text prompts. Building upon this, researchers have extended 2D text-to-image diffusion models into the 3D domain for generating human textures (UV Maps). However, some important problems about UV Map Generative models are still not solved, i.e., how to generate personalized texture maps for any given face image, and how to define and evaluate the quality of these generated texture maps. To solve the above problems, we introduce a novel method, UVMap-ID, which is a controllable and personalized UV Map generative model. Unlike traditional large-scale training methods in 2D, we propose to fine-tune a pre-trained text-to-image diffusion model which is integrated with a face fusion module for achieving ID-driven customized generation. To support the finetuning strategy, we introduce a small-scale attribute-balanced training dataset, including high-quality textures with labeled text and Face ID. Additionally, we introduce some metrics to evaluate the multiple aspects of the textures. Finally, both quantitative and qualitative analyses demonstrate the effectiveness of our method in controllable and personalized UV Map generation. Code is publicly available via https://github.com/twowwj/UVMap-ID.

Related papers

• Revisiting the Role of Texture in 3D Person Re-identification [38.1484941424058]
  This study introduces a new framework for 3D person re-identification (re-ID) We propose a method to emphasize texture in 3D person re-ID models by incorporating UVTexture mapping. In particular, the visualization and explanation are achieved through activation maps and attribute-based attention maps.
  arXiv  Detail & Related papers  (2024-10-01T02:47:34Z)
• UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling [71.87807614875497]
  We propose UV Gaussians, which models the 3D human body by jointly learning mesh deformations and 2D UV-space Gaussian textures. We collect and process a new dataset of human motion, which includes multi-view images, scanned models, parametric model registration, and corresponding texture maps. Experimental results demonstrate that our method achieves state-of-the-art synthesis of novel view and novel pose.
  arXiv  Detail & Related papers  (2024-03-18T09:03:56Z)
• Nuvo: Neural UV Mapping for Unruly 3D Representations [61.87715912587394]
  Existing UV mapping algorithms operate on geometry produced by state-of-the-art 3D reconstruction and generation techniques. We present a UV mapping method designed to operate on geometry produced by 3D reconstruction and generation techniques.
  arXiv  Detail & Related papers  (2023-12-11T18:58:38Z)
• Texture Generation on 3D Meshes with Point-UV Diffusion [86.69672057856243]
  We present Point-UV diffusion, a coarse-to-fine pipeline that marries the denoising diffusion model with UV mapping to generate high-quality texture images in UV space. Our method can process meshes of any genus, generating diversified, geometry-compatible, and high-fidelity textures.
  arXiv  Detail & Related papers  (2023-08-21T06:20:54Z)
• FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction [46.3392612457273]
  This dataset contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches. Experiments show that our method improves the reconstruction accuracy over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-25T03:21:05Z)
• AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis [78.17671694498185]
  We propose AUV-Net which learns to embed 3D surfaces into a 2D aligned UV space. As a result, textures are aligned across objects, and can thus be easily synthesized by generative models of images. The learned UV mapping and aligned texture representations enable a variety of applications including texture transfer, texture synthesis, and textured single view 3D reconstruction.
  arXiv  Detail & Related papers  (2022-04-06T21:39:24Z)
• Weakly-Supervised Photo-realistic Texture Generation for 3D Face Reconstruction [48.952656891182826]
  High-fidelity 3D face texture generation has yet to be studied. Model consists of a UV sampler and a UV generator. Training is based on pseudo ground truth blended by the 3DMM texture and the input face texture.
  arXiv  Detail & Related papers  (2021-06-14T12:34:35Z)
• StyleUV: Diverse and High-fidelity UV Map Generative Model [24.982824840625216]
  We present a novel UV map generative model that learns to generate diverse and realistic synthetic UV maps without requiring high-quality UV maps for training. Both quantitative and qualitative evaluations demonstrate that our proposed texture model produces more diverse and higher fidelity textures compared to existing methods.
  arXiv  Detail & Related papers  (2020-11-25T17:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.