Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis

• URL: http://arxiv.org/abs/2412.08603v2
• Date: Thu, 12 Dec 2024 05:04:09 GMT
• Title: Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis
• Authors: Feng Zhou, Ruiyang Liu, Chen Liu, Gaofeng He, Yong-Lu Li, Xiaogang Jin, Huamin Wang,
• Abstract summary: We propose a novel sewing pattern generation approach based on Large Multimodal Models (LMMs) LMM offers an intuitive interface for interpreting diverse design inputs. pattern-making programs could serve as well-structured and semantically meaningful representations of sewing patterns.
• Score: 27.1965932507935
• License:
• Abstract: Sewing patterns, the essential blueprints for fabric cutting and tailoring, act as a crucial bridge between design concepts and producible garments. However, existing uni-modal sewing pattern generation models struggle to effectively encode complex design concepts with a multi-modal nature and correlate them with vectorized sewing patterns that possess precise geometric structures and intricate sewing relations. In this work, we propose a novel sewing pattern generation approach Design2GarmentCode based on Large Multimodal Models (LMMs), to generate parametric pattern-making programs from multi-modal design concepts. LMM offers an intuitive interface for interpreting diverse design inputs, while pattern-making programs could serve as well-structured and semantically meaningful representations of sewing patterns, and act as a robust bridge connecting the cross-domain pattern-making knowledge embedded in LMMs with vectorized sewing patterns. Experimental results demonstrate that our method can flexibly handle various complex design expressions such as images, textual descriptions, designer sketches, or their combinations, and convert them into size-precise sewing patterns with correct stitches. Compared to previous methods, our approach significantly enhances training efficiency, generation quality, and authoring flexibility. Our code and data will be publicly available.

Related papers

• Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation [52.13927859375693]
  We propose SewingLDM, a multi-modal generative model that generates sewing patterns controlled by text prompts, body shapes, and garment sketches. To learn the sewing pattern distribution in the latent space, we design a two-step training strategy. Comprehensive qualitative and quantitative experiments show the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2024-12-19T02:05:28Z)
• AIpparel: A Large Multimodal Generative Model for Digital Garments [71.12933771326279]
  We introduce AIpparel, a large multimodal model for generating and editing sewing patterns. Our model fine-tunes state-of-the-art large multimodal models on a custom-curated large-scale dataset of over 120,000 unique garments. We propose a novel tokenization scheme that concisely encodes these complex sewing patterns so that LLMs can learn to predict them efficiently.
  arXiv  Detail & Related papers  (2024-12-05T07:35:19Z)
• GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts [53.568057283934714]
  We propose a VLM-based framework that generates content-aware text logo layouts. We introduce two model techniques to reduce the computation for processing multiple glyph images simultaneously. To support instruction-tuning of out model, we construct two extensive text logo datasets, which are 5x more larger than the existing public dataset.
  arXiv  Detail & Related papers  (2024-11-18T10:04:10Z)
• Multi-Garment Customized Model Generation [3.1679243514285194]
  Multi-Garment Customized Model Generation is a unified framework based on Latent Diffusion Models (LDMs) Our framework supports the conditional generation of multiple garments through decoupled multi-garment feature fusion. The proposed garment encoder is a plug-and-play module that can be combined with other extension modules.
  arXiv  Detail & Related papers  (2024-08-09T17:57:33Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models [17.74292177764933]
  We propose a novel hierarchical diffusion-based framework tailored for fashion design, coined as HieraFashDiff. Our model is designed to mimic the practical fashion design workflow, by unraveling the denosing process into two successive stages. Our model supports fashion design generation and fine-grained local editing in a single framework.
  arXiv  Detail & Related papers  (2024-01-15T03:38:57Z)
• Towards Garment Sewing Pattern Reconstruction from a Single Image [76.97825595711444]
  Garment sewing pattern represents the intrinsic rest shape of a garment, and is the core for many applications like fashion design, virtual try-on, and digital avatars. We first synthesize a versatile dataset, named SewFactory, which consists of around 1M images and ground-truth sewing patterns. We then propose a two-level Transformer network called Sewformer, which significantly improves the sewing pattern prediction performance.
  arXiv  Detail & Related papers  (2023-11-07T18:59:51Z)
• Modular design patterns for neural-symbolic integration: refinement and combination [0.6853165736531939]
  We formalise aspects of the neural-symbol design patterns of van Bekkum et al. These formal notions are being implemented in the heterogeneous tool set (Hets)
  arXiv  Detail & Related papers  (2022-06-09T18:41:15Z)
• CoSE: Compositional Stroke Embeddings [52.529172734044664]
  We present a generative model for complex free-form structures such as stroke-based drawing tasks. Our approach is suitable for interactive use cases such as auto-completing diagrams.
  arXiv  Detail & Related papers  (2020-06-17T15:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.