GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details

• URL: http://arxiv.org/abs/2405.12420v1
• Date: Mon, 20 May 2024 23:54:28 GMT
• Title: GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details
• Authors: Boqian Li, Xuan Li, Ying Jiang, Tianyi Xie, Feng Gao, Huamin Wang, Yin Yang, Chenfanfu Jiang,
• Abstract summary: Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and time-consuming processes. We propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate 3D garment from text prompts.
• Score: 31.92583566128599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and texturing, which are time-consuming and costly. Recent advances in diffusion-based generative models have enabled new possibilities for 3D garment generation from text prompts, images, and videos. However, existing methods either suffer from inconsistencies among multi-view images or require additional processes to separate cloth from the underlying human model. In this paper, we propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate wearable, simulation-ready 3D garment meshes from text prompts. In contrast to using multi-view images directly predicted by generative models as guidance, our 3DGS guidance ensures consistent optimization in both garment deformation and texture synthesis. Our method introduces a novel garment augmentation module, guided by normal and RGBA information, and employs implicit Neural Texture Fields (NeTF) combined with Score Distillation Sampling (SDS) to generate diverse geometric and texture details. We validate the effectiveness of our approach through comprehensive qualitative and quantitative experiments, showcasing the superior performance of GarmentDreamer over state-of-the-art alternatives. Our project page is available at: https://xuan-li.github.io/GarmentDreamerDemo/.

Related papers

• ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians [13.196912161879936]
  ClotheDreamer is a method for generating wearable, production-ready 3D garment assets from text prompts. We propose a novel representation Disentangled Clothe Gaussian Splatting (DCGS) to enable separate optimization.
  arXiv  Detail & Related papers  (2024-06-24T17:25:39Z)
• EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion [5.158983929861116]
  We present EucliDreamer, a simple and effective method to generate textures for 3D models given text and prompts. The texture is parametized as an implicit function on the 3D surface, which is optimized with the Score Distillation Sampling (SDS) process and differentiable rendering.
  arXiv  Detail & Related papers  (2024-04-16T04:44:16Z)
• RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion [39.03289977892935]
  RealmDreamer is a technique for generation of general forward-facing 3D scenes from text descriptions. Our technique does not require video or multi-view data and can synthesize a variety of high-quality 3D scenes in different styles.
  arXiv  Detail & Related papers  (2024-04-10T17:57:41Z)
• Garment3DGen: 3D Garment Stylization and Texture Generation [11.836357439129301]
  Garment3DGen is a new method to synthesize 3D garment assets from a base mesh given a single input image as guidance. We leverage the recent progress of image-to-3D diffusion methods to generate 3D garment geometries. We generate high-fidelity texture maps that are globally and locally consistent and faithfully capture the input guidance.
  arXiv  Detail & Related papers  (2024-03-27T17:59:33Z)
• WordRobe: Text-Guided Generation of Textured 3D Garments [30.614451083408266]
  "WordRobe" is a novel framework for the generation of unposed & textured 3D garment meshes from user-friendly text prompts. We demonstrate superior performance over current SOTAs for learning 3D garment latent space, garment synthesis, and text-driven texture synthesis.
  arXiv  Detail & Related papers  (2024-03-26T09:44:34Z)
• 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation [51.64796781728106]
  We propose a generative refinement network to synthesize new contents with higher quality by exploiting the natural image prior to 2D diffusion model and the global 3D information of the current scene. Our approach supports wide variety of scene generation and arbitrary camera trajectories with improved visual quality and 3D consistency.
  arXiv  Detail & Related papers  (2024-03-14T14:31:22Z)
• ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models [65.22994156658918]
  We present a method that learns to generate multi-view images in a single denoising process from real-world data. We design an autoregressive generation that renders more 3D-consistent images at any viewpoint.
  arXiv  Detail & Related papers  (2024-03-04T07:57:05Z)
• TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion [64.49276500129092]
  TextureDreamer is an image-guided texture synthesis method. It can transfer relightable textures from a small number of input images to target 3D shapes across arbitrary categories.
  arXiv  Detail & Related papers  (2024-01-17T18:55:49Z)
• TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models [77.85129451435704]
  We present a new method to synthesize textures for 3D, using large-scale-guided image diffusion models. Specifically, we leverage latent diffusion models, apply the set denoising model and aggregate denoising text map.
  arXiv  Detail & Related papers  (2023-10-20T19:15:29Z)
• Guide3D: Create 3D Avatars from Text and Image Guidance [55.71306021041785]
  Guide3D is a text-and-image-guided generative model for 3D avatar generation based on diffusion models. Our framework produces topologically and structurally correct geometry and high-resolution textures.
  arXiv  Detail & Related papers  (2023-08-18T17:55:47Z)
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images [72.15855070133425]
  We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
  arXiv  Detail & Related papers  (2022-09-22T17:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.