Semi-Supervised Visual Representation Learning for Fashion Compatibility

• URL: http://arxiv.org/abs/2109.08052v1
• Date: Thu, 16 Sep 2021 15:35:38 GMT
• Title: Semi-Supervised Visual Representation Learning for Fashion Compatibility
• Authors: Ambareesh Revanur, Vijay Kumar, Deepthi Sharma
• Abstract summary: We propose a semi-supervised learning approach to create pseudo-positive and pseudo-negative outfits on the fly during training. For each labeled outfit in a training batch, we obtain a pseudo-outfit by matching each item in the labeled outfit with unlabeled items. We conduct extensive experiments on Polyvore, Polyvore-D and our newly created large-scale Fashion Outfits datasets.
• Score: 17.893627646979038
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We consider the problem of complementary fashion prediction. Existing approaches focus on learning an embedding space where fashion items from different categories that are visually compatible are closer to each other. However, creating such labeled outfits is intensive and also not feasible to generate all possible outfit combinations, especially with large fashion catalogs. In this work, we propose a semi-supervised learning approach where we leverage large unlabeled fashion corpus to create pseudo-positive and pseudo-negative outfits on the fly during training. For each labeled outfit in a training batch, we obtain a pseudo-outfit by matching each item in the labeled outfit with unlabeled items. Additionally, we introduce consistency regularization to ensure that representation of the original images and their transformations are consistent to implicitly incorporate colour and other important attributes through self-supervision. We conduct extensive experiments on Polyvore, Polyvore-D and our newly created large-scale Fashion Outfits datasets, and show that our approach with only a fraction of labeled examples performs on-par with completely supervised methods.

Related papers

• Lost Your Style? Navigating with Semantic-Level Approach for Text-to-Outfit Retrieval [2.07180164747172]
  We introduce a groundbreaking approach to fashion recommendations: text-to-outfit retrieval task that generates a complete outfit set based solely on textual descriptions. Our model is devised at three semantic levels-item, style, and outfit-where each level progressively aggregates data to form a coherent outfit recommendation. Using the Maryland Polyvore and Polyvore Outfit datasets, our approach significantly outperformed state-of-the-art models in text-video retrieval tasks.
  arXiv  Detail & Related papers  (2023-11-03T07:23:21Z)
• FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning [66.38951790650887]
  Multimodal tasks in the fashion domain have significant potential for e-commerce. We propose a novel fashion-specific pre-training framework based on weakly-supervised triplets constructed from fashion image-text pairs. We show the triplet-based tasks are an effective addition to standard multimodal pre-training tasks.
  arXiv  Detail & Related papers  (2022-10-26T21:01:19Z)
• Learning Fashion Compatibility from In-the-wild Images [6.591937706757015]
  We propose to learn representations for compatibility prediction from in-the-wild street fashion images through self-supervised learning. Our pretext task is formulated such that the representations of different items worn by the same person are closer compared to those worn by other people. We conduct experiments on two popular fashion compatibility benchmarks - Polyvore and Polyvore-Disjoint outfits.
  arXiv  Detail & Related papers  (2022-06-13T09:05:25Z)
• OutfitTransformer: Learning Outfit Representations for Fashion Recommendation [6.890771095769622]
  We present a framework, OutfitTransformer, that learns effective outfit-level representations encoding the compatibility relationships between all items in the entire outfit. For compatibility prediction, we design an outfit token to capture a global outfit representation and train the framework using a classification loss. For complementary item retrieval, we design a target item token that additionally takes the target item specification into consideration. The generated target item embedding is then used to retrieve compatible items that match the rest of the outfit.
  arXiv  Detail & Related papers  (2022-04-11T00:55:40Z)
• Recommendation of Compatible Outfits Conditioned on Style [22.03522251199042]
  This work aims to generate outfits conditional on styles or themes as one would dress in real life. We use a novel style encoder network that renders outfit styles in a smooth latent space.
  arXiv  Detail & Related papers  (2022-03-30T09:23:32Z)
• Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing [85.74977256940855]
  We propose an Arbitrary Virtual Try-On Network (AVTON) for all-type clothes. AVTON can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person. Our approach can achieve better performance compared with the state-of-the-art virtual try-on methods.
  arXiv  Detail & Related papers  (2021-11-24T08:59:56Z)
• SMPLicit: Topology-aware Generative Model for Clothed People [65.84665248796615]
  We introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry. In the experimental section, we demonstrate SMPLicit can be readily used for fitting 3D scans and for 3D reconstruction in images of dressed people.
  arXiv  Detail & Related papers  (2021-03-11T18:57:03Z)
• Apparel-invariant Feature Learning for Apparel-changed Person Re-identification [70.16040194572406]
  Most public ReID datasets are collected in a short time window in which persons' appearance rarely changes. In real-world applications such as in a shopping mall, the same person's clothing may change, and different persons may wearing similar clothes. It is critical to learn an apparel-invariant person representation under cases like cloth changing or several persons wearing similar clothes.
  arXiv  Detail & Related papers  (2020-08-14T03:49:14Z)
• Self-supervised Visual Attribute Learning for Fashion Compatibility [71.73414832639698]
  We present an SSL framework that enables us to learn color and texture-aware features without requiring any labels during training. Our approach consists of three self-supervised tasks designed to capture different concepts that are neglected in prior work. We show that our approach can be used for transfer learning, demonstrating that we can train on one dataset while achieving high performance on a different dataset.
  arXiv  Detail & Related papers  (2020-08-01T21:53:22Z)
• Fashion Recommendation and Compatibility Prediction Using Relational Network [18.13692056232815]
  We develop a Relation Network (RN) to develop new compatibility learning models. FashionRN learns the compatibility of an entire outfit, with an arbitrary number of items, in an arbitrary order. We evaluate our model using a large dataset of 49,740 outfits that we collected from Polyvore website.
  arXiv  Detail & Related papers  (2020-05-13T21:00:54Z)
• Learning Diverse Fashion Collocation by Neural Graph Filtering [78.9188246136867]
  We propose a novel fashion collocation framework, Neural Graph Filtering, that models a flexible set of fashion items via a graph neural network. By applying symmetric operations on the edge vectors, this framework allows varying numbers of inputs/outputs and is invariant to their ordering. We evaluate the proposed approach on three popular benchmarks, the Polyvore dataset, the Polyvore-D dataset, and our reorganized Amazon Fashion dataset.
  arXiv  Detail & Related papers  (2020-03-11T16:17:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.