Fine-Grained Customized Fashion Design with Image-into-Prompt benchmark and dataset from LMM

• URL: http://arxiv.org/abs/2509.09324v1
• Date: Thu, 11 Sep 2025 10:14:36 GMT
• Title: Fine-Grained Customized Fashion Design with Image-into-Prompt benchmark and dataset from LMM
• Authors: Hui Li, Yi You, Qiqi Chen, Bingfeng Zhang, George Q. Huang,
• Abstract summary: Current generation AI models magically transform brainstorming into fancy designs easily, but the fine-grained customization still suffers from text uncertainty.<n>We propose the Better Understanding Generation (Bug) workflow with LMM to automatically create and fine-grain customize the cloth designs from chat with image-into-prompt.<n>Our framework unleashes users' creative potential beyond words and also lowers the barriers of clothing design/editing without further human involvement.
• Score: 16.82751074846868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI evolves the execution of complex workflows in industry, where the large multimodal model empowers fashion design in the garment industry. Current generation AI models magically transform brainstorming into fancy designs easily, but the fine-grained customization still suffers from text uncertainty without professional background knowledge from end-users. Thus, we propose the Better Understanding Generation (BUG) workflow with LMM to automatically create and fine-grain customize the cloth designs from chat with image-into-prompt. Our framework unleashes users' creative potential beyond words and also lowers the barriers of clothing design/editing without further human involvement. To prove the effectiveness of our model, we propose a new FashionEdit dataset that simulates the real-world clothing design workflow, evaluated from generation similarity, user satisfaction, and quality. The code and dataset: https://github.com/detectiveli/FashionEdit.

Related papers

• Rethinking Layered Graphic Design Generation with a Top-Down Approach [76.33538798060326]
  Graphic design is crucial for conveying ideas and messages. Designers usually organize their work into objects, backgrounds, and vectorized text layers to simplify editing.<n>With the rise of GenAI methods, an endless supply of high-quality graphic designs in pixel format has become more accessible.<n>Despite this, non-layered designs still inspire human designers, influencing their choices in layouts and text styles, ultimately guiding the creation of layered designs.<n>Motivated by this observation, we propose Accordion, a graphic design generation framework taking the first attempt to convert AI-generated designs into editable layered designs.
  arXiv  Detail & Related papers  (2025-07-08T02:26:08Z)
• AIpparel: A Multimodal Foundation Model for Digital Garments [71.12933771326279]
  We introduce AIpparel, a multimodal foundation model for generating and editing sewing patterns.<n>Our model fine-tunes state-of-the-art large multimodal models on a custom-curated large-scale dataset of over 120,000 unique garments.<n>We propose a novel tokenization scheme that concisely encodes these complex sewing patterns so that LLMs can learn to predict them efficiently.
  arXiv  Detail & Related papers  (2024-12-05T07:35:19Z)
• GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts [53.568057283934714]
  We propose a Vision-Language Model (VLM)-based framework that generates content-aware text logo layouts.<n>We introduce two model techniques that reduce the computational cost for processing multiple glyph images simultaneously.<n>To support instruction tuning of our model, we construct two extensive text logo datasets that are five times larger than existing public datasets.
  arXiv  Detail & Related papers  (2024-11-18T10:04:10Z)
• PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM [58.67882997399021]
  Our research introduces a unified framework for automated graphic layout generation.<n>Our data-driven method employs structured text (JSON format) and visual instruction tuning to generate layouts.<n>We develop an automated text-to-poster system that generates editable posters based on users' design intentions.
  arXiv  Detail & Related papers  (2024-06-05T03:05:52Z)
• FashionSD-X: Multimodal Fashion Garment Synthesis using Latent Diffusion [11.646594594565098]
  This study introduces a novel generative pipeline designed to transform the fashion design process by employing latent diffusion models. We leverage and enhance state-of-the-art virtual try-on datasets, including Multimodal Dress Code and VITON-HD, by integrating sketch data.
  arXiv  Detail & Related papers  (2024-04-26T14:59:42Z)
• DressCode: Autoregressively Sewing and Generating Garments from Text Guidance [61.48120090970027]
  DressCode aims to democratize design for novices and offer immense potential in fashion design, virtual try-on, and digital human creation. We first introduce SewingGPT, a GPT-based architecture integrating cross-attention with text-conditioned embedding to generate sewing patterns. We then tailor a pre-trained Stable Diffusion to generate tile-based Physically-based Rendering (PBR) textures for the garments.
  arXiv  Detail & Related papers  (2024-01-29T16:24:21Z)
• HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models [17.74292177764933]
  We propose a novel hierarchical diffusion-based framework tailored for fashion design, coined as HieraFashDiff.<n>Our model is designed to mimic the practical fashion design workflow, by unraveling the denosing process into two successive stages.<n>Our model supports fashion design generation and fine-grained local editing in a single framework.
  arXiv  Detail & Related papers  (2024-01-15T03:38:57Z)
• FashionTex: Controllable Virtual Try-on with Text and Texture [29.7855591607239]
  We propose a multi-modal interactive setting by combining the advantages of both text and texture for multi-level fashion manipulation. FashionTex framework can semantically control cloth types and local texture patterns without annotated pairwise training data.
  arXiv  Detail & Related papers  (2023-05-08T04:10:36Z)
• Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing [40.70752781891058]
  We propose the task of multimodal-conditioned fashion image editing, guiding the generation of human-centric fashion images. We tackle this problem by proposing a new architecture based on latent diffusion models. Given the lack of existing datasets suitable for the task, we also extend two existing fashion datasets.
  arXiv  Detail & Related papers  (2023-04-04T18:03:04Z)
• LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer [80.61492265221817]
  Graphic layout designs play an essential role in visual communication. Yet handcrafting layout designs is skill-demanding, time-consuming, and non-scalable to batch production. Generative models emerge to make design automation scalable but it remains non-trivial to produce designs that comply with designers' desires.
  arXiv  Detail & Related papers  (2022-12-19T21:57:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.