FashionComposer: Compositional Fashion Image Generation

• URL: http://arxiv.org/abs/2412.14168v2
• Date: Thu, 19 Dec 2024 11:59:46 GMT
• Title: FashionComposer: Compositional Fashion Image Generation
• Abstract summary: We present FashionComposer for compositional fashion image generation.<n>It takes multi-modal input (i.e., text prompt, parametric human model, garment image, and face image) and supports personalizing the appearance, pose, and figure of the human.<n>As a comprehensive solution, FashionComposer also supports many other applications like human album generation, diverse virtual try-on tasks, etc.
• Score: 38.34543351663932
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present FashionComposer for compositional fashion image generation. Unlike previous methods, FashionComposer is highly flexible. It takes multi-modal input (i.e., text prompt, parametric human model, garment image, and face image) and supports personalizing the appearance, pose, and figure of the human and assigning multiple garments in one pass. To achieve this, we first develop a universal framework capable of handling diverse input modalities. We construct scaled training data to enhance the model's robust compositional capabilities. To accommodate multiple reference images (garments and faces) seamlessly, we organize these references in a single image as an "asset library" and employ a reference UNet to extract appearance features. To inject the appearance features into the correct pixels in the generated result, we propose subject-binding attention. It binds the appearance features from different "assets" with the corresponding text features. In this way, the model could understand each asset according to their semantics, supporting arbitrary numbers and types of reference images. As a comprehensive solution, FashionComposer also supports many other applications like human album generation, diverse virtual try-on tasks, etc.

Related papers

• Grounding Free-Form Instructions for Fashion Complementary Image Generation [48.32670047915891]
  Fashion complementary image generation (CIG) aims to create garments that stylistically match a seed item based on user intent.<n>We introduce fashion complementary image generation with free-form instructions, a multimodal language-grounding setting.<n>We instantiate the task with StyleFlow, a Rectified Flow Matching model that jointly conditions on the seed image and instruction.
  arXiv  Detail & Related papers  (2026-08-24T14:26:54Z)
• Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On [50.93194436640413]
  Oxygen-TryOn is a unified foundation model for any-item virtual try-on.<n>It is built for try-on through a dedicated data engine and try-on-specific training.
  arXiv  Detail & Related papers  (2026-07-23T17:45:56Z)
• ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation [39.34778197087224]
  We introduce a new paradigm for attribute-specific image prompting, in which distinct sets of reference images are used to guide the generation of individual aspects of human appearance.<n>Our method encodes these inputs into attribute-specific tokens, which are injected into a pre-trained text-to-image diffusion model.<n>This enables compositional and disentangled control over multiple visual factors, even across multiple people within a single image.
  arXiv  Detail & Related papers  (2025-09-22T17:59:30Z)
• Fashion-RAG: Multimodal Fashion Image Editing via Retrieval-Augmented Generation [36.66066619847558]
  Fashion industry has increasingly adopted AI technologies to enhance customer experience. Fashion-RAG is first work to introduce a retrieval-augmented generation approach specifically tailored for multimodal fashion image editing.
  arXiv  Detail & Related papers  (2025-04-18T18:02:33Z)
• ComposeAnyone: Controllable Layout-to-Human Generation with Decoupled Multimodal Conditions [74.30040551058319]
  ComposeAnyone is a controllable layout-to-human generation method with decoupled multimodal conditions. Our dataset provides decoupled text and reference image annotations for different components of each human image. Experiments on multiple datasets demonstrate that ComposeAnyone generates human images with better alignment to given layouts.
  arXiv  Detail & Related papers  (2025-01-21T14:32:47Z)
• From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generation [19.096741614175524]
  Parts2Whole is a novel framework designed for generating customized portraits from multiple reference images. We first develop a semantic-aware appearance encoder to retain details of different human parts. Second, our framework supports multi-image conditioned generation through a shared self-attention mechanism.
  arXiv  Detail & Related papers  (2024-04-23T17:56:08Z)
• Instruct-Imagen: Image Generation with Multi-modal Instruction [90.04481955523514]
  instruct-imagen is a model that tackles heterogeneous image generation tasks and generalizes across unseen tasks. We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision. Human evaluation on various image generation datasets reveals that instruct-imagen matches or surpasses prior task-specific models in-domain.
  arXiv  Detail & Related papers  (2024-01-03T19:31:58Z)
• Emu: Generative Pretraining in Multimodality [43.759593451544546]
  Transformer-based multimodal foundation model can seamlessly generate images and texts in multimodal context. Emu can serve as a generalist multimodal interface for both image-to-text and text-to-image tasks. Emu demonstrates superb performance compared to state-of-the-art large multimodal models.
  arXiv  Detail & Related papers  (2023-07-11T12:45:39Z)
• Personalized Image Enhancement Featuring Masked Style Modeling [42.400427631514596]
  We enhance input images for each user based on the user's preferred images. We propose a method named masked style modeling, which can predict a style for an input image considering the contents. We conduct quantitative evaluations and a user study, and our method trained using our training scheme successfully achieves content-aware personalization.
  arXiv  Detail & Related papers  (2023-06-15T17:59:02Z)
• Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing [40.70752781891058]
  We propose the task of multimodal-conditioned fashion image editing, guiding the generation of human-centric fashion images. We tackle this problem by proposing a new architecture based on latent diffusion models. Given the lack of existing datasets suitable for the task, we also extend two existing fashion datasets.
  arXiv  Detail & Related papers  (2023-04-04T18:03:04Z)
• HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation [73.3790833537313]
  Controllable person image generation promotes a wide range of applications such as digital human interaction and virtual try-on. We propose HumanDiffusion, a coarse-to-fine alignment diffusion framework, for text-driven person image generation.
  arXiv  Detail & Related papers  (2022-11-11T14:30:34Z)
• FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning [66.38951790650887]
  Multimodal tasks in the fashion domain have significant potential for e-commerce. We propose a novel fashion-specific pre-training framework based on weakly-supervised triplets constructed from fashion image-text pairs. We show the triplet-based tasks are an effective addition to standard multimodal pre-training tasks.
  arXiv  Detail & Related papers  (2022-10-26T21:01:19Z)
• HumanGAN: A Generative Model of Humans Images [78.6284090004218]
  We present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style. Our model encodes part-based latent appearance vectors in a normalized pose-independent space and warps them to different poses, it preserves body and clothing appearance under varying posture.
  arXiv  Detail & Related papers  (2021-03-11T19:00:38Z)
• Generating Person Images with Appearance-aware Pose Stylizer [66.44220388377596]
  We present a novel end-to-end framework to generate realistic person images based on given person poses and appearances. The core of our framework is a novel generator called Appearance-aware Pose Stylizer (APS) which generates human images by coupling the target pose with the conditioned person appearance progressively.
  arXiv  Detail & Related papers  (2020-07-17T15:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.