Grounding Free-Form Instructions for Fashion Complementary Image Generation

• URL: http://arxiv.org/abs/2608.23302v1
• Date: Mon, 24 Aug 2026 14:26:54 GMT
• Title: Grounding Free-Form Instructions for Fashion Complementary Image Generation
• Abstract summary: Fashion complementary image generation (CIG) aims to create garments that stylistically match a seed item based on user intent.<n>We introduce fashion complementary image generation with free-form instructions, a multimodal language-grounding setting.<n>We instantiate the task with StyleFlow, a Rectified Flow Matching model that jointly conditions on the seed image and instruction.
• Score: 48.32670047915891
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fashion complementary image generation (CIG) aims to create garments that stylistically match a seed item based on user intent, making it a natural multimodal grounding problem where models must interpret language in visual context. Existing CIG benchmarks rely on rigid template prompts (e.g., "a photo of a skirt"), failing to reflect natural user queries and obscuring model behavior across levels of linguistic specificity. We introduce fashion complementary image generation with free-form instructions, a multimodal language-grounding setting where a model generates a compatible garment from a seed image and a natural-language instruction. To this end, we enrich three CIG benchmarks with low-, medium-, and high-specificity instructions generated by a vision-language model and validated by human annotators. We instantiate the task with StyleFlow, a Rectified Flow Matching model that jointly conditions on the seed image and instruction within a single multimodal transformer. Across image quality metrics, catalog-alignment analysis, ablations, and human evaluation, StyleFlow consistently produces instruction-aligned and stylistically coherent garments while reducing architectural complexity and inference cost relative to auxiliary-module approaches.

Related papers

• Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval [3.1033110046835457]
  In the fashion domain, this task requires understanding subtle variations such as color, pattern, and texture.<n>Existing approaches face limitations due to scarce annotated data and simplistic negative sampling.<n>We propose a novel framework that integrates a multi-grained large language model (LLaVA) to generate attribute-aware triplets.
  arXiv  Detail & Related papers  (2026-06-18T01:19:32Z)
• Images in Sentences: Scaling Interleaved Instructions for Unified Visual Generation [19.404516863003057]
  INSET is a unified generation model that seamlessly embeds images as native vocabulary within textual instructions.<n>We introduce a scalable data engine that synthesizes 15M high-quality interleaved samples from standard image and video datasets.<n>Results on InterleaveBench demonstrate that INSET significantly outperforms state-of-the-art methods in multi-image consistency and text alignment.
  arXiv  Detail & Related papers  (2026-05-12T15:54:49Z)
• UM-Text: A Unified Multimodal Model for Image Understanding [11.870303482927541]
  We propose a unified multimodal model for context understanding and visual text editing by natural language instructions.<n>We introduce a Visual Language Model (VLM) to process the instruction and reference image.<n>We propose a regional consistency loss to offer more effective supervision for glyph generation on both latent and RGB space.
  arXiv  Detail & Related papers  (2026-01-13T08:18:49Z)
• DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition [69.10628479553709]
  We introduce DRC, a novel personalized image generation framework that enhances Large Multimodal Models (LMMs)<n> DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively.<n>It involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation.
  arXiv  Detail & Related papers  (2025-04-24T08:10:10Z)
• Translatotron-V(ison): An End-to-End Model for In-Image Machine Translation [81.45400849638347]
  In-image machine translation (IIMT) aims to translate an image containing texts in source language into an image containing translations in target language. In this paper, we propose an end-to-end IIMT model consisting of four modules. Our model achieves competitive performance compared to cascaded models with only 70.9% of parameters, and significantly outperforms the pixel-level end-to-end IIMT model.
  arXiv  Detail & Related papers  (2024-07-03T08:15:39Z)
• Training-Free Consistent Text-to-Image Generation [80.4814768762066]
  Text-to-image models can portray the same subject across diverse prompts. Existing approaches fine-tune the model to teach it new words that describe specific user-provided subjects. We present ConsiStory, a training-free approach that enables consistent subject generation by sharing the internal activations of the pretrained model.
  arXiv  Detail & Related papers  (2024-02-05T18:42:34Z)
• Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization [56.12990759116612]
  Pick-and-Draw is a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods. The proposed approach can be applied to any personalized diffusion models and requires as few as a single reference image.
  arXiv  Detail & Related papers  (2024-01-30T05:56:12Z)
• Instruct-Imagen: Image Generation with Multi-modal Instruction [90.04481955523514]
  instruct-imagen is a model that tackles heterogeneous image generation tasks and generalizes across unseen tasks. We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision. Human evaluation on various image generation datasets reveals that instruct-imagen matches or surpasses prior task-specific models in-domain.
  arXiv  Detail & Related papers  (2024-01-03T19:31:58Z)
• Customize StyleGAN with One Hand Sketch [0.0]
  We propose a framework to control StyleGAN imagery with a single user sketch. We learn a conditional distribution in the latent space of a pre-trained StyleGAN model via energy-based learning. Our model can generate multi-modal images semantically aligned with the input sketch.
  arXiv  Detail & Related papers  (2023-10-29T09:32:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.