Garment Attribute Manipulation with Multi-level Attention

• URL: http://arxiv.org/abs/2409.10206v1
• Date: Mon, 16 Sep 2024 11:55:45 GMT
• Title: Garment Attribute Manipulation with Multi-level Attention
• Authors: Vittorio Casula, Lorenzo Berlincioni, Luca Cultrera, Federico Becattini, Chiara Pero, Carmen Bisogni, Marco Bertini, Alberto Del Bimbo,
• Abstract summary: We propose GAMMA, a framework that integrates attribute-disentangled representations with a multi-stage attention-based architecture. By leveraging a dual-encoder Transformer and memory block, our model achieves state-of-the-art performance on popular datasets like Shopping100k and DeepFashion.
• Score: 29.34962693598485
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the rapidly evolving field of online fashion shopping, the need for more personalized and interactive image retrieval systems has become paramount. Existing methods often struggle with precisely manipulating specific garment attributes without inadvertently affecting others. To address this challenge, we propose GAMMA (Garment Attribute Manipulation with Multi-level Attention), a novel framework that integrates attribute-disentangled representations with a multi-stage attention-based architecture. GAMMA enables targeted manipulation of fashion image attributes, allowing users to refine their searches with high accuracy. By leveraging a dual-encoder Transformer and memory block, our model achieves state-of-the-art performance on popular datasets like Shopping100k and DeepFashion.

Related papers

• ARMADA: Attribute-Based Multimodal Data Augmentation [93.05614922383822]
  Attribute-based Multimodal Data Augmentation (ARMADA) is a novel multimodal data augmentation method via knowledge-guided manipulation of visual attributes. ARMADA is a novel multimodal data generation framework that: (i) extracts knowledge-grounded attributes from symbolic KBs for semantically consistent yet distinctive image-text pair generation. This also highlights the need to leverage external knowledge proxies for enhanced interpretability and real-world grounding.
  arXiv  Detail & Related papers  (2024-08-19T15:27:25Z)
• IMAGDressing-v1: Customizable Virtual Dressing [58.44155202253754]
  IMAGDressing-v1 is a virtual dressing task that generates freely editable human images with fixed garments and optional conditions. IMAGDressing-v1 incorporates a garment UNet that captures semantic features from CLIP and texture features from VAE. We present a hybrid attention module, including a frozen self-attention and a trainable cross-attention, to integrate garment features from the garment UNet into a frozen denoising UNet.
  arXiv  Detail & Related papers  (2024-07-17T16:26:30Z)
• MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion Generation [70.83668869857665]
  MMTryon is a multi-modal multi-reference VIrtual Try-ON framework. It can generate high-quality compositional try-on results by taking a text instruction and multiple garment images as inputs.
  arXiv  Detail & Related papers  (2024-05-01T11:04:22Z)
• U-VAP: User-specified Visual Appearance Personalization via Decoupled Self Augmentation [18.841473623776153]
  State-of-the-art personalization models tend to overfit the whole subject and cannot disentangle visual characteristics in pixel space. A novel decoupled self-augmentation strategy is proposed to generate target-related and non-target samples to learn user-specified visual attributes. Experiments on various kinds of visual attributes with SOTA personalization methods show the ability of the proposed method to mimic target visual appearance in novel contexts.
  arXiv  Detail & Related papers  (2024-03-29T15:20:34Z)
• Attribute-Aware Deep Hashing with Self-Consistency for Large-Scale Fine-Grained Image Retrieval [65.43522019468976]
  We propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes. We develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors. Our models are equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities.
  arXiv  Detail & Related papers  (2023-11-21T08:20:38Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• DETR-based Layered Clothing Segmentation and Fine-Grained Attribute Recognition [10.924683447616273]
  A new DEtection TRansformer (DETR) based method is proposed to segment and recognize fine-grained attributes of ensemble clothing instances with high accuracy. We train our model on the Fashionpedia dataset and demonstrate our method surpasses SOTA models in tasks of layered clothing segmentation and fine-grained attribute recognition.
  arXiv  Detail & Related papers  (2023-04-17T09:34:48Z)
• MMFL-Net: Multi-scale and Multi-granularity Feature Learning for Cross-domain Fashion Retrieval [3.7045939497992917]
  Cross-domain fashion retrieval is a difficult task due to a wide range of consumer-to-shop (C2S) domain discrepancies. We propose a novel multi-scale and multi-granularity feature learning network (MMFL-Net), which can jointly learn global-local aggregation feature representations of clothing images. Our proposed model also combines the multi-task attribute recognition and classification module with multi-label semantic attributes and product ID labels.
  arXiv  Detail & Related papers  (2022-10-27T02:25:52Z)
• Leveraging Off-the-shelf Diffusion Model for Multi-attribute Fashion Image Manipulation [27.587905673112473]
  Fashion attribute editing is a task that aims to convert the semantic attributes of a given fashion image while preserving the irrelevant regions. Previous works typically employ conditional GANs where the generator explicitly learns the target attributes and directly execute the conversion. We explore the classifier-guided diffusion that leverages the off-the-shelf diffusion model pretrained on general visual semantics such as Imagenet.
  arXiv  Detail & Related papers  (2022-10-12T02:21:18Z)
• SMILE: Semantically-guided Multi-attribute Image and Layout Editing [154.69452301122175]
  Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs) We present a multimodal representation that handles all attributes, be it guided by random noise or images, while only using the underlying domain information of the target domain. Our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space.
  arXiv  Detail & Related papers  (2020-10-05T20:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.