Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis

• URL: http://arxiv.org/abs/2412.08603v2
• Date: Thu, 12 Dec 2024 05:04:09 GMT
• Title: Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis
• Authors: Feng Zhou, Ruiyang Liu, Chen Liu, Gaofeng He, Yong-Lu Li, Xiaogang Jin, Huamin Wang,
• Abstract summary: We propose a novel sewing pattern generation approach based on Large Multimodal Models (LMMs)<n>LMM offers an intuitive interface for interpreting diverse design inputs.<n> pattern-making programs could serve as well-structured and semantically meaningful representations of sewing patterns.
• Score: 27.1965932507935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sewing patterns, the essential blueprints for fabric cutting and tailoring, act as a crucial bridge between design concepts and producible garments. However, existing uni-modal sewing pattern generation models struggle to effectively encode complex design concepts with a multi-modal nature and correlate them with vectorized sewing patterns that possess precise geometric structures and intricate sewing relations. In this work, we propose a novel sewing pattern generation approach Design2GarmentCode based on Large Multimodal Models (LMMs), to generate parametric pattern-making programs from multi-modal design concepts. LMM offers an intuitive interface for interpreting diverse design inputs, while pattern-making programs could serve as well-structured and semantically meaningful representations of sewing patterns, and act as a robust bridge connecting the cross-domain pattern-making knowledge embedded in LMMs with vectorized sewing patterns. Experimental results demonstrate that our method can flexibly handle various complex design expressions such as images, textual descriptions, designer sketches, or their combinations, and convert them into size-precise sewing patterns with correct stitches. Compared to previous methods, our approach significantly enhances training efficiency, generation quality, and authoring flexibility. Our code and data will be publicly available.

Related papers

• GarmentDiffusion: 3D Garment Sewing Pattern Generation with Multimodal Diffusion Transformers [9.228577662928673]
  generative modeling of sewing patterns is crucial for creating diversified garments. We present textbftextitGarmentDiffusion, a new generative model capable of producing centimeter-precise, vectorized 3D sewing patterns.
  arXiv  Detail & Related papers  (2025-04-30T09:56:59Z)
• GarmageNet: A Dataset and Scalable Representation for Generic Garment Modeling [31.086617193645022]
  Garmage is a neural-network-and-CG-friendly representation for complex multi-layered garments. GarmageNet produces detailed garments with body-conforming initial geometries and intricate sewing patterns. We release an industrial-standard, large-scale, high-fidelity garment dataset.
  arXiv  Detail & Related papers  (2025-04-02T08:37:32Z)
• ChatGarment: Garment Estimation, Generation and Editing via Large Language Models [79.46056192947924]
  ChatGarment is a novel approach that leverages large vision-language models (VLMs) to automate the estimation, generation, and editing of 3D garments. It can estimate sewing patterns from in-the-wild images or sketches, generate them from text descriptions, and edit garments based on user instructions.
  arXiv  Detail & Related papers  (2024-12-23T18:59:28Z)
• Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation [52.13927859375693]
  We propose SewingLDM, a multi-modal generative model that generates sewing patterns controlled by text prompts, body shapes, and garment sketches. To learn the sewing pattern distribution in the latent space, we design a two-step training strategy. Comprehensive qualitative and quantitative experiments show the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2024-12-19T02:05:28Z)
• AIpparel: A Large Multimodal Generative Model for Digital Garments [71.12933771326279]
  We introduce AIpparel, a large multimodal model for generating and editing sewing patterns.<n>Our model fine-tunes state-of-the-art large multimodal models on a custom-curated large-scale dataset of over 120,000 unique garments.<n>We propose a novel tokenization scheme that concisely encodes these complex sewing patterns so that LLMs can learn to predict them efficiently.
  arXiv  Detail & Related papers  (2024-12-05T07:35:19Z)
• GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts [53.568057283934714]
  We propose a VLM-based framework that generates content-aware text logo layouts. We introduce two model techniques to reduce the computation for processing multiple glyph images simultaneously. To support instruction-tuning of out model, we construct two extensive text logo datasets, which are 5x more larger than the existing public dataset.
  arXiv  Detail & Related papers  (2024-11-18T10:04:10Z)
• PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM [58.67882997399021]
  Our research introduces a unified framework for automated graphic layout generation.<n>Our data-driven method employs structured text (JSON format) and visual instruction tuning to generate layouts.<n>We develop an automated text-to-poster system that generates editable posters based on users' design intentions.
  arXiv  Detail & Related papers  (2024-06-05T03:05:52Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• Hierarchical Fashion Design with Multi-stage Diffusion Models [17.848891542772446]
  Cross-modal fashion synthesis and editing offer intelligent support to fashion designers. Current diffusion models demonstrate commendable stability and controllability in image synthesis. We propose HieraFashDiff,a novel fashion design method using the shared multi-stage diffusion model.
  arXiv  Detail & Related papers  (2024-01-15T03:38:57Z)
• Towards Garment Sewing Pattern Reconstruction from a Single Image [76.97825595711444]
  Garment sewing pattern represents the intrinsic rest shape of a garment, and is the core for many applications like fashion design, virtual try-on, and digital avatars. We first synthesize a versatile dataset, named SewFactory, which consists of around 1M images and ground-truth sewing patterns. We then propose a two-level Transformer network called Sewformer, which significantly improves the sewing pattern prediction performance.
  arXiv  Detail & Related papers  (2023-11-07T18:59:51Z)
• Modular design patterns for neural-symbolic integration: refinement and combination [0.6853165736531939]
  We formalise aspects of the neural-symbol design patterns of van Bekkum et al. These formal notions are being implemented in the heterogeneous tool set (Hets)
  arXiv  Detail & Related papers  (2022-06-09T18:41:15Z)
• CoSE: Compositional Stroke Embeddings [52.529172734044664]
  We present a generative model for complex free-form structures such as stroke-based drawing tasks. Our approach is suitable for interactive use cases such as auto-completing diagrams.
  arXiv  Detail & Related papers  (2020-06-17T15:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.