Attr2Style: A Transfer Learning Approach for Inferring Fashion Styles via Apparel Attributes

• URL: http://arxiv.org/abs/2008.11662v2
• Date: Fri, 11 Dec 2020 12:03:48 GMT
• Title: Attr2Style: A Transfer Learning Approach for Inferring Fashion Styles via Apparel Attributes
• Abstract summary: We propose a transfer-learning based image captioning model that is trained on a source dataset with sufficient attribute-based ground-truth captions. We leverage this fact and train our model in an encoder-decoder based framework using attention mechanism. A Proof Of Concept for our model is under pilot at Myntra where it is exposed to some internal users for feedback.
• Score: 7.330801562698453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Popular fashion e-commerce platforms mostly provide details about low-level attributes of an apparel (eg, neck type, dress length, collar type) on their product detail pages. However, customers usually prefer to buy apparel based on their style information, or simply put, occasion (eg, party/ sports/ casual wear). Application of a supervised image-captioning model to generate style-based image captions is limited because obtaining ground-truth annotations in the form of style-based captions is difficult. This is because annotating style-based captions requires a certain amount of fashion domain expertise, and also adds to the costs and manual effort. On the contrary, low-level attribute based annotations are much more easily available. To address this issue, we propose a transfer-learning based image captioning model that is trained on a source dataset with sufficient attribute-based ground-truth captions, and used to predict style-based captions on a target dataset. The target dataset has only a limited amount of images with style-based ground-truth captions. The main motivation of our approach comes from the fact that most often there are correlations among the low-level attributes and the higher-level styles for an apparel. We leverage this fact and train our model in an encoder-decoder based framework using attention mechanism. In particular, the encoder of the model is first trained on the source dataset to obtain latent representations capturing the low-level attributes. The trained model is fine-tuned to generate style-based captions for the target dataset. To highlight the effectiveness of our method, we qualitatively and quantitatively demonstrate that the captions generated by our approach are close to the actual style information for the evaluated apparel. A Proof Of Concept for our model is under pilot at Myntra where it is exposed to some internal users for feedback.

Related papers

• RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributes [5.735692008673119]
  Fashion Image Captioning (FIC) plays a vital role in enhancing user experience and product search in e-commerce platforms.<n>We propose RA-CoA, a training-free framework that disentangles fashion image captioning into two interpretable stages.<n> RA-CoA significantly improves caption quality, achieving an average gain of 26.3% METEOR score over zero-shot captioning.
  arXiv  Detail & Related papers  (2026-09-12T18:51:24Z)
• Low-Rank Head Avatar Personalization with Registers [36.7667914190956]
  We introduce a novel method for low-rank personalization of a generic model for head avatar generation.<n>Our approach faithfully captures unseen faces, outperforming existing methods quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2025-06-02T17:53:14Z)
• Masked Language Prompting for Generative Data Augmentation in Few-shot Fashion Style Recognition [1.03590082373586]
  A dataset for fashion style recognition is challenging due to the inherent subjectivity and ambiguity of style concepts. Recent advances in text-to-image models have facilitated generative data augmentation by synthesizing images from labeled data. We propose textbfMasked Language Prompting (MLP), a novel prompting strategy that masks selected words in a reference caption and leverages large language models to generate diverse yet semantically coherents.
  arXiv  Detail & Related papers  (2025-04-28T03:42:42Z)
• Measuring Style Similarity in Diffusion Models [118.22433042873136]
  We present a framework for understanding and extracting style descriptors from images. Our framework comprises a new dataset curated using the insight that style is a subjective property of an image. We also propose a method to extract style attribute descriptors that can be used to style of a generated image to the images used in the training dataset of a text-to-image model.
  arXiv  Detail & Related papers  (2024-04-01T17:58:30Z)
• ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer [57.6482608202409]
  Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. We introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
  arXiv  Detail & Related papers  (2023-08-29T17:36:02Z)
• Visual Captioning at Will: Describing Images and Videos Guided by a Few Stylized Sentences [49.66987347397398]
  Few-Shot Stylized Visual Captioning aims to generate captions in any desired style, using only a few examples as guidance during inference. We propose a framework called FS-StyleCap for this task, which utilizes a conditional encoder-decoder language model and a visual projection module.
  arXiv  Detail & Related papers  (2023-07-31T04:26:01Z)
• Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion [17.99150939602917]
  State-of-The-Art (SoTA) image captioning models often rely on the Microsoft COCO (MS-COCO) dataset for training. We present a novel approach to address previous challenges by showcasing how captions generated from different SoTA models can be effectively fused.
  arXiv  Detail & Related papers  (2023-06-20T15:13:02Z)
• Leveraging Off-the-shelf Diffusion Model for Multi-attribute Fashion Image Manipulation [27.587905673112473]
  Fashion attribute editing is a task that aims to convert the semantic attributes of a given fashion image while preserving the irrelevant regions. Previous works typically employ conditional GANs where the generator explicitly learns the target attributes and directly execute the conversion. We explore the classifier-guided diffusion that leverages the off-the-shelf diffusion model pretrained on general visual semantics such as Imagenet.
  arXiv  Detail & Related papers  (2022-10-12T02:21:18Z)
• Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation [120.96012935286913]
  We propose a novel adversarial style augmentation approach, which can generate hard stylized images during training. Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains.
  arXiv  Detail & Related papers  (2022-07-11T14:01:25Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets [56.018551958004814]
  This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources. Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision. We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
  arXiv  Detail & Related papers  (2021-11-24T19:00:05Z)
• Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewards [41.04264950157818]
  The goal of this work is to develop a novel learning framework for accurate and expressive fashion captioning. We seed the description of an item by first identifying its attributes, and introduce attribute-level semantic (ALS) reward and sentence-level semantic (SLS) reward as metrics to improve the quality of text descriptions.
  arXiv  Detail & Related papers  (2020-08-06T14:52:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.