Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions

• URL: http://arxiv.org/abs/2306.07520v4
• Date: Sun, 31 Dec 2023 16:54:05 GMT
• Title: Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions
• Authors: Weizhen He and Yiheng Deng and Shixiang Tang and Qihao Chen and Qingsong Xie and Yizhou Wang and Lei Bai and Feng Zhu and Rui Zhao and Wanli Ouyang and Donglian Qi and Yunfeng Yan
• Abstract summary: We propose a new instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Our instruct-ReID is a more general ReID setting, where existing 6 ReID tasks can be viewed as special cases by designing different instructions. Experimental results show that the proposed multi-purpose ReID model, trained on our OmniReID benchmark without fine-tuning, can improve +0.5%, +0.6%, +7.7% mAP on Market1501, MSMT17, CUHK03 for traditional ReID, +6.4%, +7.1%, +11.2% mAP on PRCC,
• Score: 64.55715112644562
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human intelligence can retrieve any person according to both visual and language descriptions. However, the current computer vision community studies specific person re-identification (ReID) tasks in different scenarios separately, which limits the applications in the real world. This paper strives to resolve this problem by proposing a new instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Our instruct-ReID is a more general ReID setting, where existing 6 ReID tasks can be viewed as special cases by designing different instructions. We propose a large-scale OmniReID benchmark and an adaptive triplet loss as a baseline method to facilitate research in this new setting. Experimental results show that the proposed multi-purpose ReID model, trained on our OmniReID benchmark without fine-tuning, can improve +0.5%, +0.6%, +7.7% mAP on Market1501, MSMT17, CUHK03 for traditional ReID, +6.4%, +7.1%, +11.2% mAP on PRCC, VC-Clothes, LTCC for clothes-changing ReID, +11.7% mAP on COCAS+ real2 for clothes template based clothes-changing ReID when using only RGB images, +24.9% mAP on COCAS+ real2 for our newly defined language-instructed ReID, +4.3% on LLCM for visible-infrared ReID, +2.6% on CUHK-PEDES for text-to-image ReID. The datasets, the model, and code will be available at https://github.com/hwz-zju/Instruct-ReID.

Related papers

• Multiple Information Prompt Learning for Cloth-Changing Person Re-Identification [17.948263914620238]
  We propose a novel multiple information prompt learning (MIPL) scheme for cloth-changing person ReID. TheCIS module is designed to decouple clothing information from the original RGB image features. The Bio-guided attention (BGA) module is proposed to increase the learning intensity of the model for key information.
  arXiv  Detail & Related papers  (2024-11-01T03:08:10Z)
• Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification [62.894790379098005]
  We propose a novel instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Instruct-ReID is the first exploration of a general ReID setting, where existing 6 ReID tasks can be viewed as special cases by assigning different instructions. We propose a novel baseline model, IRM, with an adaptive triplet loss to handle various retrieval tasks within a unified framework.
  arXiv  Detail & Related papers  (2024-05-28T03:35:46Z)
• AaP-ReID: Improved Attention-Aware Person Re-identification [2.5761958263376745]
  AaP-ReID is a more effective method for person ReID that incorporates channel-wise attention into a ResNet-based architecture. Our method incorporates the Channel-Wise Attention Bottleneck block and can learn discriminating features by dynamically adjusting the importance ofeach channel in the feature maps.
  arXiv  Detail & Related papers  (2023-09-27T16:54:38Z)
• Body Part-Based Representation Learning for Occluded Person Re-Identification [102.27216744301356]
  Occluded person re-identification (ReID) is a person retrieval task which aims at matching occluded person images with holistic ones. Part-based methods have been shown beneficial as they offer fine-grained information and are well suited to represent partially visible human bodies. We propose BPBreID, a body part-based ReID model for solving the above issues.
  arXiv  Detail & Related papers  (2022-11-07T16:48:41Z)
• Clothes-Changing Person Re-identification with RGB Modality Only [102.44387094119165]
  We propose a Clothes-based Adrial Loss (CAL) to mine clothes-irrelevant features from the original RGB images. Videos contain richer appearance and additional temporal information, which can be used to model propertemporal patterns.
  arXiv  Detail & Related papers  (2022-04-14T11:38:28Z)
• Unsupervised Pre-training for Person Re-identification [90.98552221699508]
  We present a large scale unlabeled person re-identification (Re-ID) dataset "LUPerson" We make the first attempt of performing unsupervised pre-training for improving the generalization ability of the learned person Re-ID feature representation.
  arXiv  Detail & Related papers  (2020-12-07T14:48:26Z)
• Fine-Grained Re-Identification [1.8275108630751844]
  This paper proposes a computationally efficient fine-grained ReID model, FGReID, which is among the first models to unify image and video ReID. FGReID takes advantage of video-based pre-training and spatial feature attention to improve performance on both video and image ReID tasks.
  arXiv  Detail & Related papers  (2020-11-26T21:04:17Z)
• Apparel-invariant Feature Learning for Apparel-changed Person Re-identification [70.16040194572406]
  Most public ReID datasets are collected in a short time window in which persons' appearance rarely changes. In real-world applications such as in a shopping mall, the same person's clothing may change, and different persons may wearing similar clothes. It is critical to learn an apparel-invariant person representation under cases like cloth changing or several persons wearing similar clothes.
  arXiv  Detail & Related papers  (2020-08-14T03:49:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.