Nuvo: Neural UV Mapping for Unruly 3D Representations

• URL: http://arxiv.org/abs/2312.05283v1
• Date: Mon, 11 Dec 2023 18:58:38 GMT
• Title: Nuvo: Neural UV Mapping for Unruly 3D Representations
• Authors: Pratul P. Srinivasan and Stephan J. Garbin and Dor Verbin and Jonathan T. Barron and Ben Mildenhall
• Abstract summary: Existing UV mapping algorithms operate on geometry produced by state-of-the-art 3D reconstruction and generation techniques. We present a UV mapping method designed to operate on geometry produced by 3D reconstruction and generation techniques.
• Score: 61.87715912587394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing UV mapping algorithms are designed to operate on well-behaved meshes, instead of the geometry representations produced by state-of-the-art 3D reconstruction and generation techniques. As such, applying these methods to the volume densities recovered by neural radiance fields and related techniques (or meshes triangulated from such fields) results in texture atlases that are too fragmented to be useful for tasks such as view synthesis or appearance editing. We present a UV mapping method designed to operate on geometry produced by 3D reconstruction and generation techniques. Instead of computing a mapping defined on a mesh's vertices, our method Nuvo uses a neural field to represent a continuous UV mapping, and optimizes it to be a valid and well-behaved mapping for just the set of visible points, i.e. only points that affect the scene's appearance. We show that our model is robust to the challenges posed by ill-behaved geometry, and that it produces editable UV mappings that can represent detailed appearance.

Related papers

• Semantic UV mapping to improve texture inpainting for indoor scenes [0.0]
  This work aims to improve texture inpainting after clutter removal in scanned indoor meshes. This is achieved with a new UV mapping pre-processing step which leverages semantic information of indoor scenes to more accurately match the UV islands with the 3D representation of distinct structural elements like walls and floors.
  arXiv  Detail & Related papers  (2024-07-12T13:21:25Z)
• UVMap-ID: A Controllable and Personalized UV Map Generative Model [67.71022515856653]
  We introduce UVMap-ID, a controllable and personalized UV Map generative model. Unlike traditional large-scale training methods in 2D, we propose to fine-tune a pre-trained text-to-image diffusion model. Both quantitative and qualitative analyses demonstrate the effectiveness of our method in controllable and personalized UV Map generation.
  arXiv  Detail & Related papers  (2024-04-22T20:30:45Z)
• UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling [71.87807614875497]
  We propose UV Gaussians, which models the 3D human body by jointly learning mesh deformations and 2D UV-space Gaussian textures. We collect and process a new dataset of human motion, which includes multi-view images, scanned models, parametric model registration, and corresponding texture maps. Experimental results demonstrate that our method achieves state-of-the-art synthesis of novel view and novel pose.
  arXiv  Detail & Related papers  (2024-03-18T09:03:56Z)
• Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing [79.10630153776759]
  3D Gaussian splatting, emerging as a groundbreaking approach, has drawn increasing attention for its capabilities of high-fidelity reconstruction and real-time rendering. We propose a novel approach, namely Texture-GS, to disentangle the appearance from the geometry by representing it as a 2D texture mapped onto the 3D surface. Our method not only facilitates high-fidelity appearance editing but also achieves real-time rendering on consumer-level devices.
  arXiv  Detail & Related papers  (2024-03-15T06:42:55Z)
• Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement [78.48648360358193]
  We present a novel framework that generates textured surface meshes from images. Our approach begins by efficiently initializing the geometry and view-dependency appearance with a NeRF. We jointly refine the appearance with geometry and bake it into texture images for real-time rendering.
  arXiv  Detail & Related papers  (2023-03-03T17:14:44Z)
• FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction [46.3392612457273]
  This dataset contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches. Experiments show that our method improves the reconstruction accuracy over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-25T03:21:05Z)
• AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis [78.17671694498185]
  We propose AUV-Net which learns to embed 3D surfaces into a 2D aligned UV space. As a result, textures are aligned across objects, and can thus be easily synthesized by generative models of images. The learned UV mapping and aligned texture representations enable a variety of applications including texture transfer, texture synthesis, and textured single view 3D reconstruction.
  arXiv  Detail & Related papers  (2022-04-06T21:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.