DETR-based Layered Clothing Segmentation and Fine-Grained Attribute Recognition

• URL: http://arxiv.org/abs/2304.08107v1
• Date: Mon, 17 Apr 2023 09:34:48 GMT
• Title: DETR-based Layered Clothing Segmentation and Fine-Grained Attribute Recognition
• Authors: Hao Tian, Yu Cao, P. Y. Mok
• Abstract summary: A new DEtection TRansformer (DETR) based method is proposed to segment and recognize fine-grained attributes of ensemble clothing instances with high accuracy. We train our model on the Fashionpedia dataset and demonstrate our method surpasses SOTA models in tasks of layered clothing segmentation and fine-grained attribute recognition.
• Score: 10.924683447616273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clothing segmentation and fine-grained attribute recognition are challenging tasks at the crossing of computer vision and fashion, which segment the entire ensemble clothing instances as well as recognize detailed attributes of the clothing products from any input human images. Many new models have been developed for the tasks in recent years, nevertheless the segmentation accuracy is less than satisfactory in case of layered clothing or fashion products in different scales. In this paper, a new DEtection TRansformer (DETR) based method is proposed to segment and recognize fine-grained attributes of ensemble clothing instances with high accuracy. In this model, we propose a \textbf{multi-layered attention module} by aggregating features of different scales, determining the various scale components of a single instance, and merging them together. We train our model on the Fashionpedia dataset and demonstrate our method surpasses SOTA models in tasks of layered clothing segmentation and fine-grained attribute recognition.

Related papers

• Garment Attribute Manipulation with Multi-level Attention [29.34962693598485]
  We propose GAMMA, a framework that integrates attribute-disentangled representations with a multi-stage attention-based architecture. By leveraging a dual-encoder Transformer and memory block, our model achieves state-of-the-art performance on popular datasets like Shopping100k and DeepFashion.
  arXiv  Detail & Related papers  (2024-09-16T11:55:45Z)
• Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification [78.52704557647438]
  We propose a novel FIne-grained Representation and Recomposition (FIRe$2$) framework to tackle both limitations without any auxiliary annotation or data. Experiments demonstrate that FIRe$2$ can achieve state-of-the-art performance on five widely-used cloth-changing person Re-ID benchmarks.
  arXiv  Detail & Related papers  (2023-08-21T12:59:48Z)
• Data Efficient Training with Imbalanced Label Sample Distribution for Fashion Detection [5.912870746288055]
  We propose a state-of-the-art weighted objective function to boost the performance of deep neural networks (DNNs) for multi-label classification with long-tailed data distribution. Our experiments involve image-based attribute classification of fashion apparels, and the results demonstrate favorable performance for the new weighting method.
  arXiv  Detail & Related papers  (2023-05-07T21:25:09Z)
• Leveraging Off-the-shelf Diffusion Model for Multi-attribute Fashion Image Manipulation [27.587905673112473]
  Fashion attribute editing is a task that aims to convert the semantic attributes of a given fashion image while preserving the irrelevant regions. Previous works typically employ conditional GANs where the generator explicitly learns the target attributes and directly execute the conversion. We explore the classifier-guided diffusion that leverages the off-the-shelf diffusion model pretrained on general visual semantics such as Imagenet.
  arXiv  Detail & Related papers  (2022-10-12T02:21:18Z)
• Fashionformer: A simple, Effective and Unified Baseline for Human Fashion Segmentation and Recognition [80.74495836502919]
  In this work, we focus on joint human fashion segmentation and attribute recognition. We introduce the object query for segmentation and the attribute query for attribute prediction. For attribute stream, we design a novel Multi-Layer Rendering module to explore more fine-grained features.
  arXiv  Detail & Related papers  (2022-04-10T11:11:10Z)
• Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing [85.74977256940855]
  We propose an Arbitrary Virtual Try-On Network (AVTON) for all-type clothes. AVTON can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person. Our approach can achieve better performance compared with the state-of-the-art virtual try-on methods.
  arXiv  Detail & Related papers  (2021-11-24T08:59:56Z)
• Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning [71.74073012364326]
  We propose an Attribute-Specific Embedding Network (ASEN) to jointly learn multiple attribute-specific embeddings. The proposed ASEN is comprised of a global branch and a local branch. Experiments on three fashion-related datasets, i.e., FashionAI, DARN, and DeepFashion, show the effectiveness of ASEN for fine-grained fashion similarity prediction.
  arXiv  Detail & Related papers  (2021-04-06T11:26:38Z)
• Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset [62.77342894987297]
  We propose a novel Attribute-Mask RCNN model to jointly perform instance segmentation and localized attribute recognition. We also demonstrate instance segmentation models pre-trained on Fashionpedia achieve better transfer learning performance on other fashion datasets than ImageNet pre-training.
  arXiv  Detail & Related papers  (2020-04-26T02:38:26Z)
• Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network [59.479783847922135]
  We propose an Attribute-Specific Embedding Network (ASEN) to jointly learn multiple attribute-specific embeddings in an end-to-end manner. ASEN is able to locate the related regions and capture the essential patterns under the guidance of the specified attribute. Experiments on four fashion-related datasets show the effectiveness of ASEN for fine-grained fashion similarity learning.
  arXiv  Detail & Related papers  (2020-02-07T14:42:26Z)
• TailorGAN: Making User-Defined Fashion Designs [28.805686791183618]
  We propose a novel self-supervised model to synthesize garment images with disentangled attributes without paired data. Our method consists of a reconstruction learning step and an adversarial learning step. Experiments on this dataset and real-world samples demonstrate that our method can synthesize much better results than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-17T16:54:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.