Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation

• URL: http://arxiv.org/abs/2412.14453v1
• Date: Thu, 19 Dec 2024 02:05:28 GMT
• Title: Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation
• Authors: Shengqi Liu, Yuhao Cheng, Zhuo Chen, Xingyu Ren, Wenhan Zhu, Lincheng Li, Mengxiao Bi, Xiaokang Yang, Yichao Yan,
• Abstract summary: We propose SewingLDM, a multi-modal generative model that generates sewing patterns controlled by text prompts, body shapes, and garment sketches. To learn the sewing pattern distribution in the latent space, we design a two-step training strategy. Comprehensive qualitative and quantitative experiments show the effectiveness of our proposed method.
• Score: 52.13927859375693
• License:
• Abstract: Generating sewing patterns in garment design is receiving increasing attention due to its CG-friendly and flexible-editing nature. Previous sewing pattern generation methods have been able to produce exquisite clothing, but struggle to design complex garments with detailed control. To address these issues, we propose SewingLDM, a multi-modal generative model that generates sewing patterns controlled by text prompts, body shapes, and garment sketches. Initially, we extend the original vector of sewing patterns into a more comprehensive representation to cover more intricate details and then compress them into a compact latent space. To learn the sewing pattern distribution in the latent space, we design a two-step training strategy to inject the multi-modal conditions, \ie, body shapes, text prompts, and garment sketches, into a diffusion model, ensuring the generated garments are body-suited and detail-controlled. Comprehensive qualitative and quantitative experiments show the effectiveness of our proposed method, significantly surpassing previous approaches in terms of complex garment design and various body adaptability. Our project page: https://shengqiliu1.github.io/SewingLDM.

Related papers

• Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis [27.1965932507935]
  We propose a novel sewing pattern generation approach based on Large Multimodal Models (LMMs) LMM offers an intuitive interface for interpreting diverse design inputs. pattern-making programs could serve as well-structured and semantically meaningful representations of sewing patterns.
  arXiv  Detail & Related papers  (2024-12-11T18:26:45Z)
• AIpparel: A Large Multimodal Generative Model for Digital Garments [71.12933771326279]
  We introduce AIpparel, a large multimodal model for generating and editing sewing patterns. Our model fine-tunes state-of-the-art large multimodal models on a custom-curated large-scale dataset of over 120,000 unique garments. We propose a novel tokenization scheme that concisely encodes these complex sewing patterns so that LLMs can learn to predict them efficiently.
  arXiv  Detail & Related papers  (2024-12-05T07:35:19Z)
• FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images [56.63824638417697]
  FabricDiffusion is a method for transferring fabric textures from a single clothing image to 3D garments of arbitrary shapes. We show that FabricDiffusion can transfer various features from a single clothing image including texture patterns, material properties, and detailed prints and logos.
  arXiv  Detail & Related papers  (2024-10-02T17:57:12Z)
• Multi-Garment Customized Model Generation [3.1679243514285194]
  Multi-Garment Customized Model Generation is a unified framework based on Latent Diffusion Models (LDMs) Our framework supports the conditional generation of multiple garments through decoupled multi-garment feature fusion. The proposed garment encoder is a plug-and-play module that can be combined with other extension modules.
  arXiv  Detail & Related papers  (2024-08-09T17:57:33Z)
• Garment Recovery with Shape and Deformation Priors [51.41962835642731]
  We propose a method that delivers realistic garment models from real-world images, regardless of garment shape or deformation. Not only does our approach recover the garment geometry accurately, it also yields models that can be directly used by downstream applications such as animation and simulation.
  arXiv  Detail & Related papers  (2023-11-17T07:06:21Z)
• Towards Garment Sewing Pattern Reconstruction from a Single Image [76.97825595711444]
  Garment sewing pattern represents the intrinsic rest shape of a garment, and is the core for many applications like fashion design, virtual try-on, and digital avatars. We first synthesize a versatile dataset, named SewFactory, which consists of around 1M images and ground-truth sewing patterns. We then propose a two-level Transformer network called Sewformer, which significantly improves the sewing pattern prediction performance.
  arXiv  Detail & Related papers  (2023-11-07T18:59:51Z)
• Structure-Preserving 3D Garment Modeling with Neural Sewing Machines [190.70647799442565]
  We propose a novel Neural Sewing Machine (NSM), a learning-based framework for structure-preserving 3D garment modeling. NSM is capable of representing 3D garments under diverse garment shapes and topologies, realistically reconstructing 3D garments from 2D images with the preserved structure, and accurately manipulating the 3D garment categories, shapes, and topologies.
  arXiv  Detail & Related papers  (2022-11-12T16:43:29Z)
• NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments [7.331799534004012]
  We propose to use a garment sewing pattern to facilitate the intrinsic garment shape estimation. We introduce NeuralTailor, a novel architecture based on point-level attention for set regression with variable cardinality. Our experiments show that NeuralTailor successfully reconstructs sewing patterns and generalizes to garment types with pattern topologies unseen during training.
  arXiv  Detail & Related papers  (2022-01-31T08:33:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.