DressCode: Autoregressively Sewing and Generating Garments from Text Guidance

• URL: http://arxiv.org/abs/2401.16465v4
• Date: Sat, 15 Jun 2024 01:58:22 GMT
• Title: DressCode: Autoregressively Sewing and Generating Garments from Text Guidance
• Authors: Kai He, Kaixin Yao, Qixuan Zhang, Jingyi Yu, Lingjie Liu, Lan Xu,
• Abstract summary: DressCode aims to democratize design for novices and offer immense potential in fashion design, virtual try-on, and digital human creation. We first introduce SewingGPT, a GPT-based architecture integrating cross-attention with text-conditioned embedding to generate sewing patterns. We then tailor a pre-trained Stable Diffusion to generate tile-based Physically-based Rendering (PBR) textures for the garments.
• Score: 61.48120090970027
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Apparel's significant role in human appearance underscores the importance of garment digitalization for digital human creation. Recent advances in 3D content creation are pivotal for digital human creation. Nonetheless, garment generation from text guidance is still nascent. We introduce a text-driven 3D garment generation framework, DressCode, which aims to democratize design for novices and offer immense potential in fashion design, virtual try-on, and digital human creation. We first introduce SewingGPT, a GPT-based architecture integrating cross-attention with text-conditioned embedding to generate sewing patterns with text guidance. We then tailor a pre-trained Stable Diffusion to generate tile-based Physically-based Rendering (PBR) textures for the garments. By leveraging a large language model, our framework generates CG-friendly garments through natural language interaction. It also facilitates pattern completion and texture editing, streamlining the design process through user-friendly interaction. This framework fosters innovation by allowing creators to freely experiment with designs and incorporate unique elements into their work. With comprehensive evaluations and comparisons with other state-of-the-art methods, our method showcases superior quality and alignment with input prompts. User studies further validate our high-quality rendering results, highlighting its practical utility and potential in production settings. Our project page is https://IHe-KaiI.github.io/DressCode/.

Related papers

• GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details [31.92583566128599]
  Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and time-consuming processes. We propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate 3D garment from text prompts.
  arXiv  Detail & Related papers  (2024-05-20T23:54:28Z)
• Design2Cloth: 3D Cloth Generation from 2D Masks [34.80461276448817]
  We propose Design2Cloth, a high fidelity 3D generative model trained on a real world dataset from more than 2000 subject scans. Under a series of both qualitative and quantitative experiments, we showcase that Design2Cloth outperforms current state-of-the-art cloth generative models by a large margin.
  arXiv  Detail & Related papers  (2024-04-03T12:32:13Z)
• WordRobe: Text-Guided Generation of Textured 3D Garments [30.614451083408266]
  "WordRobe" is a novel framework for the generation of unposed & textured 3D garment meshes from user-friendly text prompts. We demonstrate superior performance over current SOTAs for learning 3D garment latent space, garment synthesis, and text-driven texture synthesis.
  arXiv  Detail & Related papers  (2024-03-26T09:44:34Z)
• Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text [38.591390310534024]
  We focus on automatic texture design for cartoon characters on input instructions. This is challenging for domain-specific requirements and a lack of high-quality data. We propose Make-ItVivi the first attempt to enable high-quality texture generation from text in UV.
  arXiv  Detail & Related papers  (2024-03-25T16:08:04Z)
• TADA! Text to Animatable Digital Avatars [57.52707683788961]
  TADA takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures. We derive an optimizable high-resolution body model from SMPL-X with 3D displacements and a texture map. We render normals and RGB images of the generated character and exploit their latent embeddings in the SDS training process.
  arXiv  Detail & Related papers  (2023-08-21T17:59:10Z)
• Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On [47.4550741942217]
  Cloth2Tex is a self-supervised method that generates texture maps with reasonable layout and structural consistency. It can be used to support high-fidelity texture inpainting. We evaluate our approach both qualitatively and quantitatively and demonstrate that Cloth2Tex can generate high-quality texture maps.
  arXiv  Detail & Related papers  (2023-08-08T14:32:38Z)
• Text-guided 3D Human Generation from 2D Collections [69.04031635550294]
  We introduce Text-guided 3D Human Generation (texttT3H), where a model is to generate a 3D human, guided by the fashion description. CCH adopts cross-modal attention to fuse compositional human rendering with the extracted fashion semantics. We conduct evaluations on DeepFashion and SHHQ with diverse fashion attributes covering the shape, fabric, and color of upper and lower clothing.
  arXiv  Detail & Related papers  (2023-05-23T17:50:15Z)
• Structure-Preserving 3D Garment Modeling with Neural Sewing Machines [190.70647799442565]
  We propose a novel Neural Sewing Machine (NSM), a learning-based framework for structure-preserving 3D garment modeling. NSM is capable of representing 3D garments under diverse garment shapes and topologies, realistically reconstructing 3D garments from 2D images with the preserved structure, and accurately manipulating the 3D garment categories, shapes, and topologies.
  arXiv  Detail & Related papers  (2022-11-12T16:43:29Z)
• TEMOS: Generating diverse human motions from textual descriptions [53.85978336198444]
  We address the problem of generating diverse 3D human motions from textual descriptions. We propose TEMOS, a text-conditioned generative model leveraging variational autoencoder (VAE) training with human motion data. We show that TEMOS framework can produce both skeleton-based animations as in prior work, as well more expressive SMPL body motions.
  arXiv  Detail & Related papers  (2022-04-25T14:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.