MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignment

• URL: http://arxiv.org/abs/2404.02899v2
• Date: Fri, 19 Apr 2024 18:53:41 GMT
• Title: MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignment
• Authors: Duygu Ceylan, Valentin Deschaintre, Thibault Groueix, Rosalie Martin, Chun-Hao Huang, Romain Rouffet, Vladimir Kim, Gaëtan Lassagne,
• Abstract summary: MatAtlas is a method for consistent text-guided 3D model. By proposing a multi-step texture refinement process, we significantly improve the quality and consistency.
• Score: 11.721314027024547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present MatAtlas, a method for consistent text-guided 3D model texturing. Following recent progress we leverage a large scale text-to-image generation model (e.g., Stable Diffusion) as a prior to texture a 3D model. We carefully design an RGB texturing pipeline that leverages a grid pattern diffusion, driven by depth and edges. By proposing a multi-step texture refinement process, we significantly improve the quality and 3D consistency of the texturing output. To further address the problem of baked-in lighting, we move beyond RGB colors and pursue assigning parametric materials to the assets. Given the high-quality initial RGB texture, we propose a novel material retrieval method capitalized on Large Language Models (LLM), enabling editabiliy and relightability. We evaluate our method on a wide variety of geometries and show that our method significantly outperform prior arts. We also analyze the role of each component through a detailed ablation study.

Related papers

• TexPro: Text-guided PBR Texturing with Procedural Material Modeling [23.8905505397344]
  TexPro is a novel method for high-fidelity material generation for input 3D meshes given text prompts. We first generate multi-view reference images given the input textual prompt by employing the latest text-to-image model. We derive texture maps through a rendering-based optimization with recent differentiable procedural materials.
  arXiv  Detail & Related papers  (2024-10-21T11:10:07Z)
• TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling [37.67373829836975]
  We present TexGen, a novel multi-view sampling and resampling framework for texture generation. Our proposed method produces significantly better texture quality for diverse 3D objects with a high degree of view consistency. Our proposed texture generation technique can also be applied to texture editing while preserving the original identity.
  arXiv  Detail & Related papers  (2024-08-02T14:24:40Z)
• MaPa: Text-driven Photorealistic Material Painting for 3D Shapes [80.66880375862628]
  This paper aims to generate materials for 3D meshes from text descriptions. Unlike existing methods that synthesize texture maps, we propose to generate segment-wise procedural material graphs. Our framework supports high-quality rendering and provides substantial flexibility in editing.
  arXiv  Detail & Related papers  (2024-04-26T17:54:38Z)
• EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion [5.158983929861116]
  We present EucliDreamer, a simple and effective method to generate textures for 3D models given text and prompts. The texture is parametized as an implicit function on the 3D surface, which is optimized with the Score Distillation Sampling (SDS) process and differentiable rendering.
  arXiv  Detail & Related papers  (2024-04-16T04:44:16Z)
• SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors [49.03627933561738]
  SceneTex is a novel method for generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion priors. SceneTex enables various and accurate texture synthesis for 3D-FRONT scenes, demonstrating significant improvements in visual quality and prompt fidelity over the prior texture generation methods.
  arXiv  Detail & Related papers  (2023-11-28T22:49:57Z)
• TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models [77.85129451435704]
  We present a new method to synthesize textures for 3D, using large-scale-guided image diffusion models. Specifically, we leverage latent diffusion models, apply the set denoising model and aggregate denoising text map.
  arXiv  Detail & Related papers  (2023-10-20T19:15:29Z)
• PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation [89.09455618184239]
  Recent advances in text-to-3D human generation have been groundbreaking. We propose a model called PaintHuman to address the challenges from two aspects. We use the depth map as a guidance to ensure realistic semantically aligned textures.
  arXiv  Detail & Related papers  (2023-10-14T00:37:16Z)
• Directional Texture Editing for 3D Models [51.31499400557996]
  ITEM3D is designed for automatic textbf3D object editing according to the text textbfInstructions. Leveraging the diffusion models and the differentiable rendering, ITEM3D takes the rendered images as the bridge of text and 3D representation.
  arXiv  Detail & Related papers  (2023-09-26T12:01:13Z)
• TEXTure: Text-Guided Texturing of 3D Shapes [71.13116133846084]
  We present TEXTure, a novel method for text-guided editing, editing, and transfer of textures for 3D shapes. We define a trimap partitioning process that generates seamless 3D textures without requiring explicit surface textures.
  arXiv  Detail & Related papers  (2023-02-03T13:18:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.