GenesisTex: Adapting Image Denoising Diffusion to Texture Space

• URL: http://arxiv.org/abs/2403.17782v1
• Date: Tue, 26 Mar 2024 15:15:15 GMT
• Title: GenesisTex: Adapting Image Denoising Diffusion to Texture Space
• Authors: Chenjian Gao, Boyan Jiang, Xinghui Li, Yingpeng Zhang, Qian Yu,
• Abstract summary: GenesisTex is a novel method for synthesizing textures for 3D geometries from text descriptions. We maintain a latent texture map for each viewpoint, which is updated with predicted noise on the rendering of the corresponding viewpoint. Global consistency is achieved through the integration of style consistency mechanisms within the noise prediction network.
• Score: 15.907134430301133
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present GenesisTex, a novel method for synthesizing textures for 3D geometries from text descriptions. GenesisTex adapts the pretrained image diffusion model to texture space by texture space sampling. Specifically, we maintain a latent texture map for each viewpoint, which is updated with predicted noise on the rendering of the corresponding viewpoint. The sampled latent texture maps are then decoded into a final texture map. During the sampling process, we focus on both global and local consistency across multiple viewpoints: global consistency is achieved through the integration of style consistency mechanisms within the noise prediction network, and low-level consistency is achieved by dynamically aligning latent textures. Finally, we apply reference-based inpainting and img2img on denser views for texture refinement. Our approach overcomes the limitations of slow optimization in distillation-based methods and instability in inpainting-based methods. Experiments on meshes from various sources demonstrate that our method surpasses the baseline methods quantitatively and qualitatively.

Related papers

• TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling [37.67373829836975]
  We present TexGen, a novel multi-view sampling and resampling framework for texture generation. Our proposed method produces significantly better texture quality for diverse 3D objects with a high degree of view consistency. Our proposed texture generation technique can also be applied to texture editing while preserving the original identity.
  arXiv  Detail & Related papers  (2024-08-02T14:24:40Z)
• Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis [61.189479577198846]
  We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt. Our approach fine-tunes a diffusion model on a single texture, and learns to embed that statistical distribution in the output domain of the model. At generation time, our fine-tuned diffusion model is used through a score aggregation strategy to generate output texture images of arbitrary resolution on a single GPU.
  arXiv  Detail & Related papers  (2024-05-13T21:53:09Z)
• TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion [64.49276500129092]
  TextureDreamer is an image-guided texture synthesis method. It can transfer relightable textures from a small number of input images to target 3D shapes across arbitrary categories.
  arXiv  Detail & Related papers  (2024-01-17T18:55:49Z)
• SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors [49.03627933561738]
  SceneTex is a novel method for generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion priors. SceneTex enables various and accurate texture synthesis for 3D-FRONT scenes, demonstrating significant improvements in visual quality and prompt fidelity over the prior texture generation methods.
  arXiv  Detail & Related papers  (2023-11-28T22:49:57Z)
• ConTex-Human: Free-View Rendering of Human from a Single Image with Texture-Consistent Synthesis [49.28239918969784]
  We introduce a texture-consistent back view synthesis module that could transfer the reference image content to the back view. We also propose a visibility-aware patch consistency regularization for texture mapping and refinement combined with the synthesized back view texture.
  arXiv  Detail & Related papers  (2023-11-28T13:55:53Z)
• TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models [77.85129451435704]
  We present a new method to synthesize textures for 3D, using large-scale-guided image diffusion models. Specifically, we leverage latent diffusion models, apply the set denoising model and aggregate denoising text map.
  arXiv  Detail & Related papers  (2023-10-20T19:15:29Z)
• Text2Tex: Text-driven Texture Synthesis via Diffusion Models [31.773823357617093]
  We present Text2Tex, a novel method for generating high-quality textures for 3D meshes from text prompts. Our method incorporates inpainting into a pre-trained depth-aware image diffusion model to progressively synthesize high resolution partial textures from multiple viewpoints.
  arXiv  Detail & Related papers  (2023-03-20T19:02:13Z)
• TEXTure: Text-Guided Texturing of 3D Shapes [71.13116133846084]
  We present TEXTure, a novel method for text-guided editing, editing, and transfer of textures for 3D shapes. We define a trimap partitioning process that generates seamless 3D textures without requiring explicit surface textures.
  arXiv  Detail & Related papers  (2023-02-03T13:18:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.