Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details

• URL: http://arxiv.org/abs/2308.16880v1
• Date: Thu, 31 Aug 2023 17:37:23 GMT
• Title: Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details
• Authors: Inwoo Hwang, Hyeonwoo Kim, Young Min Kim
• Abstract summary: We propose Text2Scene, a method to automatically create realistic textures for virtual scenes composed of multiple objects. Our pipeline adds detailed texture on labeled 3D geometries in the room such that the generated colors respect the hierarchical structure or semantic parts that are often composed of similar materials.
• Score: 12.660352353074012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Text2Scene, a method to automatically create realistic textures for virtual scenes composed of multiple objects. Guided by a reference image and text descriptions, our pipeline adds detailed texture on labeled 3D geometries in the room such that the generated colors respect the hierarchical structure or semantic parts that are often composed of similar materials. Instead of applying flat stylization on the entire scene at a single step, we obtain weak semantic cues from geometric segmentation, which are further clarified by assigning initial colors to segmented parts. Then we add texture details for individual objects such that their projections on image space exhibit feature embedding aligned with the embedding of the input. The decomposition makes the entire pipeline tractable to a moderate amount of computation resources and memory. As our framework utilizes the existing resources of image and text embedding, it does not require dedicated datasets with high-quality textures designed by skillful artists. To the best of our knowledge, it is the first practical and scalable approach that can create detailed and realistic textures of the desired style that maintain structural context for scenes with multiple objects.

Related papers

• RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting [34.827355403635536]
  We propose a 3D scene framework referred to as RoomTex. RoomTex generates high-fidelity and style-consistent textures for un-consistent scene meshes. We propose to maintain superior alignment between RGB and edge detection methods.
  arXiv  Detail & Related papers  (2024-06-04T16:27:09Z)
• Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text [38.591390310534024]
  We focus on automatic texture design for cartoon characters on input instructions. This is challenging for domain-specific requirements and a lack of high-quality data. We propose Make-ItVivi the first attempt to enable high-quality texture generation from text in UV.
  arXiv  Detail & Related papers  (2024-03-25T16:08:04Z)
• TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion [64.49276500129092]
  TextureDreamer is an image-guided texture synthesis method. It can transfer relightable textures from a small number of input images to target 3D shapes across arbitrary categories.
  arXiv  Detail & Related papers  (2024-01-17T18:55:49Z)
• SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors [49.03627933561738]
  SceneTex is a novel method for generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion priors. SceneTex enables various and accurate texture synthesis for 3D-FRONT scenes, demonstrating significant improvements in visual quality and prompt fidelity over the prior texture generation methods.
  arXiv  Detail & Related papers  (2023-11-28T22:49:57Z)
• DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation [31.353409149640605]
  In this paper, we propose a novel framework to generate 3D textures for immersive VR experiences. To survive, we separate texture cues in confidential regions and learn to network textures in real-world environments.
  arXiv  Detail & Related papers  (2023-10-19T19:29:23Z)
• LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts [60.54912319612113]
  Diffusion-based generative models have significantly advanced text-to-image generation but encounter challenges when processing lengthy and intricate text prompts. We present a novel approach leveraging Large Language Models (LLMs) to extract critical components from text prompts. Our evaluation on complex prompts featuring multiple objects demonstrates a substantial improvement in recall compared to baseline diffusion models.
  arXiv  Detail & Related papers  (2023-10-16T17:57:37Z)
• Semantic Image Translation for Repairing the Texture Defects of Building Models [16.764719266178655]
  We introduce a novel approach for synthesizing faccade texture images that authentically reflect the architectural style from a structured label map. Our proposed method is also capable of synthesizing texture images with specific styles for faccades that lack pre-existing textures.
  arXiv  Detail & Related papers  (2023-03-30T14:38:53Z)
• TEXTure: Text-Guided Texturing of 3D Shapes [71.13116133846084]
  We present TEXTure, a novel method for text-guided editing, editing, and transfer of textures for 3D shapes. We define a trimap partitioning process that generates seamless 3D textures without requiring explicit surface textures.
  arXiv  Detail & Related papers  (2023-02-03T13:18:45Z)
• SceneComposer: Any-Level Semantic Image Synthesis [80.55876413285587]
  We propose a new framework for conditional image synthesis from semantic layouts of any precision levels. The framework naturally reduces to text-to-image (T2I) at the lowest level with no shape information, and it becomes segmentation-to-image (S2I) at the highest level. We introduce several novel techniques to address the challenges coming with this new setup.
  arXiv  Detail & Related papers  (2022-11-21T18:59:05Z)
• Image Inpainting Guided by Coherence Priors of Semantics and Textures [62.92586889409379]
  We introduce coherence priors between the semantics and textures which make it possible to concentrate on completing separate textures in a semantic-wise manner. We also propose two coherence losses to constrain the consistency between the semantics and the inpainted image in terms of the overall structure and detailed textures.
  arXiv  Detail & Related papers  (2020-12-15T02:59:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.