CLEAR: Cross-Transformers with Pre-trained Language Model is All you need for Person Attribute Recognition and Retrieval

• URL: http://arxiv.org/abs/2403.06119v2
• Date: Tue, 30 Apr 2024 12:05:02 GMT
• Title: CLEAR: Cross-Transformers with Pre-trained Language Model is All you need for Person Attribute Recognition and Retrieval
• Authors: Doanh C. Bui, Thinh V. Le, Ba Hung Ngo, Tae Jong Choi,
• Abstract summary: Person attribute recognition and attribute-based retrieval are two core human-centric tasks. We introduce a robust cross-transformers network to handle person attribute recognition. We also introduce an effective training strategy to train only a few additional parameters for adapters. CLEAR achieves state-of-the-art performance or competitive results for both tasks.
• Score: 0.18749305679160366
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Person attribute recognition and attribute-based retrieval are two core human-centric tasks. In the recognition task, the challenge is specifying attributes depending on a person's appearance, while the retrieval task involves searching for matching persons based on attribute queries. There is a significant relationship between recognition and retrieval tasks. In this study, we demonstrate that if there is a sufficiently robust network to solve person attribute recognition, it can be adapted to facilitate better performance for the retrieval task. Another issue that needs addressing in the retrieval task is the modality gap between attribute queries and persons' images. Therefore, in this paper, we present CLEAR, a unified network designed to address both tasks. We introduce a robust cross-transformers network to handle person attribute recognition. Additionally, leveraging a pre-trained language model, we construct pseudo-descriptions for attribute queries and introduce an effective training strategy to train only a few additional parameters for adapters, facilitating the handling of the retrieval task. Finally, the unified CLEAR model is evaluated on five benchmarks: PETA, PA100K, Market-1501, RAPv2, and UPAR-2024. Without bells and whistles, CLEAR achieves state-of-the-art performance or competitive results for both tasks, significantly outperforming other competitors in terms of person retrieval performance on the widely-used Market-1501 dataset.

Related papers

• Divide and Conquer: Hybrid Pre-training for Person Search [40.13016375392472]
  We propose a hybrid pre-training framework specifically designed for person search using sub-task data only. Our model can achieve significant improvements across diverse protocols, such as person search method, fine-tuning data, pre-training data and model backbone. Our code and pre-trained models are released for plug-and-play usage to the person search community.
  arXiv  Detail & Related papers  (2023-12-13T08:33:50Z)
• Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised Learning [25.356999988217325]
  Relevance modeling aims to locate desirable items for corresponding queries. auxiliary query-item interactions extracted from user historical behavior data could provide hints to reveal users' search intents further. Our model builds multi-level co-attention for distilling coarse-grained and fine-grained semantic representations from both neighbor and target views.
  arXiv  Detail & Related papers  (2023-08-10T06:52:53Z)
• UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph [89.98762327725112]
  Multi-hop Question Answering over Knowledge Graph(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question. We propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning.
  arXiv  Detail & Related papers  (2022-12-02T04:08:09Z)
• Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning [54.66399120084227]
  Recent state-of-the-art neural text matching models (PLMs) are hard to generalize to different tasks. We adopt a specialization-generalization training strategy and refer to it as Match-Prompt. In specialization stage, descriptions of different matching tasks are mapped to only a few prompt tokens. In generalization stage, text matching model explores the essential matching signals by being trained on diverse multiple matching tasks.
  arXiv  Detail & Related papers  (2022-04-06T11:01:08Z)
• The Overlooked Classifier in Human-Object Interaction Recognition [82.20671129356037]
  We encode the semantic correlation among classes into the classification head by initializing the weights with language embeddings of HOIs. We propose a new loss named LSE-Sign to enhance multi-label learning on a long-tailed dataset. Our simple yet effective method enables detection-free HOI classification, outperforming the state-of-the-arts that require object detection and human pose by a clear margin.
  arXiv  Detail & Related papers  (2022-03-10T23:35:00Z)
• End-to-end Person Search Sequentially Trained on Aggregated Dataset [1.9766522384767227]
  We propose a new end-to-end model that jointly computes detection and feature extraction steps. We show that aggregating more pedestrian detection datasets without costly identity annotations makes the shared feature maps more generic.
  arXiv  Detail & Related papers  (2022-01-24T11:22:15Z)
• QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction [57.56700153507383]
  This paper proposes a unified query attribute value extraction system in e-commerce search named QUEACO. For the NER phase, QUEACO adopts a novel teacher-student network, where a teacher network that is trained on the strongly-labeled data generates pseudo-labels. For the AVN phase, we also leverage the weakly-labeled query-to-attribute behavior data to normalize surface form attribute values from queries into canonical forms from products.
  arXiv  Detail & Related papers  (2021-08-19T03:24:23Z)
• Multi-Attribute Enhancement Network for Person Search [7.85420914437147]
  Person Search is designed to jointly solve the problems of Person Detection and Person Re-identification (Re-ID) Visual character attributes play a key role in retrieving the query person, which has been explored in Re-ID but has been ignored in Person Search. We introduce attribute learning into the model, allowing the use of attribute features for retrieval task.
  arXiv  Detail & Related papers  (2021-02-16T05:43:47Z)
• Symbiotic Adversarial Learning for Attribute-based Person Search [86.7506832053208]
  We present a symbiotic adversarial learning framework, called SAL.Two GANs sit at the base of the framework in a symbiotic learning scheme. Specifically, two different types of generative adversarial networks learn collaboratively throughout the training process.
  arXiv  Detail & Related papers  (2020-07-19T07:24:45Z)
• Joint Item Recommendation and Attribute Inference: An Adaptive Graph Convolutional Network Approach [61.2786065744784]
  In recommender systems, users and items are associated with attributes, and users show preferences to items. As annotating user (item) attributes is a labor intensive task, the attribute values are often incomplete with many missing attribute values. We propose an Adaptive Graph Convolutional Network (AGCN) approach for joint item recommendation and attribute inference.
  arXiv  Detail & Related papers  (2020-05-25T10:50:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.