Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset

• URL: http://arxiv.org/abs/2004.12276v2
• Date: Sat, 18 Jul 2020 21:02:49 GMT
• Title: Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset
• Authors: Menglin Jia, Mengyun Shi, Mikhail Sirotenko, Yin Cui, Claire Cardie, Bharath Hariharan, Hartwig Adam, Serge Belongie
• Abstract summary: We propose a novel Attribute-Mask RCNN model to jointly perform instance segmentation and localized attribute recognition. We also demonstrate instance segmentation models pre-trained on Fashionpedia achieve better transfer learning performance on other fashion datasets than ImageNet pre-training.
• Score: 62.77342894987297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we explore the task of instance segmentation with attribute localization, which unifies instance segmentation (detect and segment each object instance) and fine-grained visual attribute categorization (recognize one or multiple attributes). The proposed task requires both localizing an object and describing its properties. To illustrate the various aspects of this task, we focus on the domain of fashion and introduce Fashionpedia as a step toward mapping out the visual aspects of the fashion world. Fashionpedia consists of two parts: (1) an ontology built by fashion experts containing 27 main apparel categories, 19 apparel parts, 294 fine-grained attributes and their relationships; (2) a dataset with everyday and celebrity event fashion images annotated with segmentation masks and their associated per-mask fine-grained attributes, built upon the Fashionpedia ontology. In order to solve this challenging task, we propose a novel Attribute-Mask RCNN model to jointly perform instance segmentation and localized attribute recognition, and provide a novel evaluation metric for the task. We also demonstrate instance segmentation models pre-trained on Fashionpedia achieve better transfer learning performance on other fashion datasets than ImageNet pre-training. Fashionpedia is available at: https://fashionpedia.github.io/home/index.html.

Related papers

• DETR-based Layered Clothing Segmentation and Fine-Grained Attribute Recognition [10.924683447616273]
  A new DEtection TRansformer (DETR) based method is proposed to segment and recognize fine-grained attributes of ensemble clothing instances with high accuracy. We train our model on the Fashionpedia dataset and demonstrate our method surpasses SOTA models in tasks of layered clothing segmentation and fine-grained attribute recognition.
  arXiv  Detail & Related papers  (2023-04-17T09:34:48Z)
• Leveraging Off-the-shelf Diffusion Model for Multi-attribute Fashion Image Manipulation [27.587905673112473]
  Fashion attribute editing is a task that aims to convert the semantic attributes of a given fashion image while preserving the irrelevant regions. Previous works typically employ conditional GANs where the generator explicitly learns the target attributes and directly execute the conversion. We explore the classifier-guided diffusion that leverages the off-the-shelf diffusion model pretrained on general visual semantics such as Imagenet.
  arXiv  Detail & Related papers  (2022-10-12T02:21:18Z)
• Fashionformer: A simple, Effective and Unified Baseline for Human Fashion Segmentation and Recognition [80.74495836502919]
  In this work, we focus on joint human fashion segmentation and attribute recognition. We introduce the object query for segmentation and the attribute query for attribute prediction. For attribute stream, we design a novel Multi-Layer Rendering module to explore more fine-grained features.
  arXiv  Detail & Related papers  (2022-04-10T11:11:10Z)
• Attribute Prototype Network for Any-Shot Learning [113.50220968583353]
  We argue that an image representation with integrated attribute localization ability would be beneficial for any-shot, i.e. zero-shot and few-shot, image classification tasks. We propose a novel representation learning framework that jointly learns global and local features using only class-level attributes.
  arXiv  Detail & Related papers  (2022-04-04T02:25:40Z)
• FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation [22.691709684780292]
  The proposed FashionSearchNet-v2 architecture is able to learn attribute specific representations by leveraging on its weakly-supervised localization module. The network is jointly trained with the combination of attribute classification and triplet ranking loss to estimate local representations. Experiments performed on several datasets that are rich in terms of the number of attributes show that FashionSearchNet-v2 outperforms the other state-of-the-art attribute manipulation techniques.
  arXiv  Detail & Related papers  (2021-11-28T13:50:20Z)
• Sentiment Analysis of Fashion Related Posts in Social Media [36.623221002330226]
  We propose a novel framework that jointly leverages the image vision, post text, as well as fashion attribute modality to determine the sentiment category. Since there is no existing dataset suitable for this task, we prepare a large-scale sentiment analysis dataset of over 12k fashion related social media posts.
  arXiv  Detail & Related papers  (2021-11-15T14:58:09Z)
• Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning [71.74073012364326]
  We propose an Attribute-Specific Embedding Network (ASEN) to jointly learn multiple attribute-specific embeddings. The proposed ASEN is comprised of a global branch and a local branch. Experiments on three fashion-related datasets, i.e., FashionAI, DARN, and DeepFashion, show the effectiveness of ASEN for fine-grained fashion similarity prediction.
  arXiv  Detail & Related papers  (2021-04-06T11:26:38Z)
• Where to Look and How to Describe: Fashion Image Retrieval with an Attentional Heterogeneous Bilinear Network [50.19558726384559]
  We propose a biologically inspired framework for image-based fashion product retrieval. Our proposed framework achieves satisfactory performance on three image-based fashion product retrieval benchmarks.
  arXiv  Detail & Related papers  (2020-10-26T06:01:09Z)
• Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network [59.479783847922135]
  We propose an Attribute-Specific Embedding Network (ASEN) to jointly learn multiple attribute-specific embeddings in an end-to-end manner. ASEN is able to locate the related regions and capture the essential patterns under the guidance of the specified attribute. Experiments on four fashion-related datasets show the effectiveness of ASEN for fine-grained fashion similarity learning.
  arXiv  Detail & Related papers  (2020-02-07T14:42:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.