NeRF-Texture: Synthesizing Neural Radiance Field Textures

• URL: http://arxiv.org/abs/2412.10004v1
• Date: Fri, 13 Dec 2024 09:41:48 GMT
• Title: NeRF-Texture: Synthesizing Neural Radiance Field Textures
• Authors: Yi-Hua Huang, Yan-Pei Cao, Yu-Kun Lai, Ying Shan, Lin Gao,
• Abstract summary: We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. We can synthesize NeRF-based textures through patch matching of latent features.
• Score: 77.24205024987414
• License:
• Abstract: Texture synthesis is a fundamental problem in computer graphics that would benefit various applications. Existing methods are effective in handling 2D image textures. In contrast, many real-world textures contain meso-structure in the 3D geometry space, such as grass, leaves, and fabrics, which cannot be effectively modeled using only 2D image textures. We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. This allows textures with meso-structure to be effectively learned as latent features situated on the base shape, which are fed into a NeRF decoder trained simultaneously to represent the rich view-dependent appearance. Using this implicit representation, we can synthesize NeRF-based textures through patch matching of latent features. However, inconsistencies between the metrics of the reconstructed content space and the latent feature space may compromise the synthesis quality. To enhance matching performance, we further regularize the distribution of latent features by incorporating a clustering constraint. In addition to generating NeRF textures over a planar domain, our method can also synthesize NeRF textures over curved surfaces, which are practically useful. Experimental results and evaluations demonstrate the effectiveness of our approach.

Related papers

• Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis [61.189479577198846]
  We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt. Our approach fine-tunes a diffusion model on a single texture, and learns to embed that statistical distribution in the output domain of the model. At generation time, our fine-tuned diffusion model is used through a score aggregation strategy to generate output texture images of arbitrary resolution on a single GPU.
  arXiv  Detail & Related papers  (2024-05-13T21:53:09Z)
• ConTex-Human: Free-View Rendering of Human from a Single Image with Texture-Consistent Synthesis [49.28239918969784]
  We introduce a texture-consistent back view synthesis module that could transfer the reference image content to the back view. We also propose a visibility-aware patch consistency regularization for texture mapping and refinement combined with the synthesized back view texture.
  arXiv  Detail & Related papers  (2023-11-28T13:55:53Z)
• Neural Texture Puppeteer: A Framework for Neural Geometry and Texture Rendering of Articulated Shapes, Enabling Re-Identification at Interactive Speed [2.8544822698499255]
  We present a neural rendering pipeline for textured articulated shapes that we call Neural Texture Puppeteer. A texture auto-encoder makes use of this information to encode textured images into a global latent code. Our method can be applied to endangered species where data is limited.
  arXiv  Detail & Related papers  (2023-11-28T10:51:05Z)
• TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models [77.85129451435704]
  We present a new method to synthesize textures for 3D, using large-scale-guided image diffusion models. Specifically, we leverage latent diffusion models, apply the set denoising model and aggregate denoising text map.
  arXiv  Detail & Related papers  (2023-10-20T19:15:29Z)
• Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement [78.48648360358193]
  We present a novel framework that generates textured surface meshes from images. Our approach begins by efficiently initializing the geometry and view-dependency appearance with a NeRF. We jointly refine the appearance with geometry and bake it into texture images for real-time rendering.
  arXiv  Detail & Related papers  (2023-03-03T17:14:44Z)
• StructNeRF: Neural Radiance Fields for Indoor Scenes with Structural Hints [23.15914545835831]
  StructNeRF is a solution to novel view synthesis for indoor scenes with sparse inputs. Our method improves both the geometry and the view synthesis performance of NeRF without any additional training on external data.
  arXiv  Detail & Related papers  (2022-09-12T14:33:27Z)
• AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis [78.17671694498185]
  We propose AUV-Net which learns to embed 3D surfaces into a 2D aligned UV space. As a result, textures are aligned across objects, and can thus be easily synthesized by generative models of images. The learned UV mapping and aligned texture representations enable a variety of applications including texture transfer, texture synthesis, and textured single view 3D reconstruction.
  arXiv  Detail & Related papers  (2022-04-06T21:39:24Z)
• Deep Tiling: Texture Tile Synthesis Using a Deep Learning Approach [0.0]
  In many cases a texture image cannot cover a large 3D model surface because of its small resolution. Deep learning based texture synthesis has proven to be very effective in such cases. We propose a novel approach to example-based texture synthesis by using a robust deep learning process.
  arXiv  Detail & Related papers  (2021-03-14T18:17:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.