SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion
Priors
- URL: http://arxiv.org/abs/2311.17261v1
- Date: Tue, 28 Nov 2023 22:49:57 GMT
- Title: SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion
Priors
- Authors: Dave Zhenyu Chen, Haoxuan Li, Hsin-Ying Lee, Sergey Tulyakov, Matthias
Nie{\ss}ner
- Abstract summary: SceneTex is a novel method for generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion priors.
SceneTex enables various and accurate texture synthesis for 3D-FRONT scenes, demonstrating significant improvements in visual quality and prompt fidelity over the prior texture generation methods.
- Score: 49.03627933561738
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose SceneTex, a novel method for effectively generating high-quality
and style-consistent textures for indoor scenes using depth-to-image diffusion
priors. Unlike previous methods that either iteratively warp 2D views onto a
mesh surface or distillate diffusion latent features without accurate geometric
and style cues, SceneTex formulates the texture synthesis task as an
optimization problem in the RGB space where style and geometry consistency are
properly reflected. At its core, SceneTex proposes a multiresolution texture
field to implicitly encode the mesh appearance. We optimize the target texture
via a score-distillation-based objective function in respective RGB renderings.
To further secure the style consistency across views, we introduce a
cross-attention decoder to predict the RGB values by cross-attending to the
pre-sampled reference locations in each instance. SceneTex enables various and
accurate texture synthesis for 3D-FRONT scenes, demonstrating significant
improvements in visual quality and prompt fidelity over the prior texture
generation methods.
Related papers
- RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting [34.827355403635536]
We propose a 3D scene framework referred to as RoomTex.
RoomTex generates high-fidelity and style-consistent textures for un-consistent scene meshes.
We propose to maintain superior alignment between RGB and edge detection methods.
arXiv Detail & Related papers (2024-06-04T16:27:09Z) - GenesisTex: Adapting Image Denoising Diffusion to Texture Space [15.907134430301133]
GenesisTex is a novel method for synthesizing textures for 3D geometries from text descriptions.
We maintain a latent texture map for each viewpoint, which is updated with predicted noise on the rendering of the corresponding viewpoint.
Global consistency is achieved through the integration of style consistency mechanisms within the noise prediction network.
arXiv Detail & Related papers (2024-03-26T15:15:15Z) - TexRO: Generating Delicate Textures of 3D Models by Recursive Optimization [54.59133974444805]
TexRO is a novel method for generating delicate textures of a known 3D mesh by optimizing its UV texture.
We demonstrate the superior performance of TexRO in terms of texture quality, detail preservation, visual consistency, and, notably runtime speed.
arXiv Detail & Related papers (2024-03-22T07:45:51Z) - Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering [47.78392889256976]
Paint-it is a text-driven high-fidelity texture map synthesis method for 3D rendering.
Paint-it synthesizes texture maps from a text description by synthesis-through-optimization, exploiting the Score-Distillation Sampling (SDS)
We show that DC-PBR inherently schedules the optimization curriculum according to texture frequency and naturally filters out the noisy signals from SDS.
arXiv Detail & Related papers (2023-12-18T17:17:08Z) - SceneWiz3D: Towards Text-guided 3D Scene Composition [134.71933134180782]
Existing approaches either leverage large text-to-image models to optimize a 3D representation or train 3D generators on object-centric datasets.
We introduce SceneWiz3D, a novel approach to synthesize high-fidelity 3D scenes from text.
arXiv Detail & Related papers (2023-12-13T18:59:30Z) - TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion
Models [77.85129451435704]
We present a new method to synthesize textures for 3D, using large-scale-guided image diffusion models.
Specifically, we leverage latent diffusion models, apply the set denoising model and aggregate denoising text map.
arXiv Detail & Related papers (2023-10-20T19:15:29Z) - Text2Tex: Text-driven Texture Synthesis via Diffusion Models [31.773823357617093]
We present Text2Tex, a novel method for generating high-quality textures for 3D meshes from text prompts.
Our method incorporates inpainting into a pre-trained depth-aware image diffusion model to progressively synthesize high resolution partial textures from multiple viewpoints.
arXiv Detail & Related papers (2023-03-20T19:02:13Z) - CompNVS: Novel View Synthesis with Scene Completion [83.19663671794596]
We propose a generative pipeline performing on a sparse grid-based neural scene representation to complete unobserved scene parts.
We process encoded image features in 3D space with a geometry completion network and a subsequent texture inpainting network to extrapolate the missing area.
Photorealistic image sequences can be finally obtained via consistency-relevant differentiable rendering.
arXiv Detail & Related papers (2022-07-23T09:03:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.