ProFD: Prompt-Guided Feature Disentangling for Occluded Person Re-Identification

• URL: http://arxiv.org/abs/2409.20081v1
• Date: Mon, 30 Sep 2024 08:31:14 GMT
• Title: ProFD: Prompt-Guided Feature Disentangling for Occluded Person Re-Identification
• Authors: Can Cui, Siteng Huang, Wenxuan Song, Pengxiang Ding, Min Zhang, Donglin Wang,
• Abstract summary: We propose a Prompt-guided Feature Disentangling method (ProFD) to generate well-aligned part features. ProFD first designs part-specific prompts and utilizes noisy segmentation mask to preliminarily align visual and textual embedding. We employ a self-distillation strategy, retaining pre-trained knowledge of CLIP to mitigate over-fitting.
• Score: 34.38227097059117
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To address the occlusion issues in person Re-Identification (ReID) tasks, many methods have been proposed to extract part features by introducing external spatial information. However, due to missing part appearance information caused by occlusion and noisy spatial information from external model, these purely vision-based approaches fail to correctly learn the features of human body parts from limited training data and struggle in accurately locating body parts, ultimately leading to misaligned part features. To tackle these challenges, we propose a Prompt-guided Feature Disentangling method (ProFD), which leverages the rich pre-trained knowledge in the textual modality facilitate model to generate well-aligned part features. ProFD first designs part-specific prompts and utilizes noisy segmentation mask to preliminarily align visual and textual embedding, enabling the textual prompts to have spatial awareness. Furthermore, to alleviate the noise from external masks, ProFD adopts a hybrid-attention decoder, ensuring spatial and semantic consistency during the decoding process to minimize noise impact. Additionally, to avoid catastrophic forgetting, we employ a self-distillation strategy, retaining pre-trained knowledge of CLIP to mitigate over-fitting. Evaluation results on the Market1501, DukeMTMC-ReID, Occluded-Duke, Occluded-ReID, and P-DukeMTMC datasets demonstrate that ProFD achieves state-of-the-art results. Our project is available at: https://github.com/Cuixxx/ProFD.

Related papers

• Coarse-to-Fine Proposal Refinement Framework for Audio Temporal Forgery Detection and Localization [60.899082019130766]
  We introduce a frame-level detection network (FDN) and a proposal refinement network (PRN) for audio temporal forgery detection and localization. FDN aims to mine informative inconsistency cues between real and fake frames to obtain discriminative features that are beneficial for roughly indicating forgery regions. PRN is responsible for predicting confidence scores and regression offsets to refine the coarse-grained proposals derived from the FDN.
  arXiv  Detail & Related papers  (2024-07-23T15:07:52Z)
• Part-Attention Based Model Make Occluded Person Re-Identification Stronger [1.7648680700685022]
  We introduce PAB-ReID, which is a novel ReID model incorporating part-attention mechanisms to tackle the issues effectively. Firstly, we introduce the human parsing label to guide the generation of more accurate human part attention maps. In addition, we propose a fine-grained feature focuser for generating fine-grained human local feature representations while suppressing background interference.
  arXiv  Detail & Related papers  (2024-04-04T13:43:11Z)
• DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery [71.6345505427213]
  DPMesh is an innovative framework for occluded human mesh recovery. It capitalizes on the profound diffusion prior about object structure and spatial relationships embedded in a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-04-01T18:59:13Z)
• Dynamic Patch-aware Enrichment Transformer for Occluded Person Re-Identification [14.219232629274186]
  We present an end-to-end solution known as the Dynamic Patch-aware Enrichment Transformer (DPEFormer) This model effectively distinguishes human body information from occlusions automatically and dynamically. To ensure that DPSM and the entire DPEFormer can effectively learn with only identity labels, we also propose a Realistic Occlusion Augmentation (ROA) strategy.
  arXiv  Detail & Related papers  (2024-02-16T03:53:30Z)
• Feature Completion Transformer for Occluded Person Re-identification [25.159974510754992]
  Occluded person re-identification (Re-ID) is a challenging problem due to the destruction of occluders. We propose a Feature Completion Transformer (FCFormer) to implicitly complement the semantic information of occluded parts in the feature space. FCFormer achieves superior performance and outperforms the state-of-the-art methods by significant margins on occluded datasets.
  arXiv  Detail & Related papers  (2023-03-03T01:12:57Z)
• Occluded Person Re-Identification via Relational Adaptive Feature Correction Learning [8.015703163954639]
  Occluded person re-identification (Re-ID) in images captured by multiple cameras is challenging because the target person is occluded by pedestrians or objects. Most existing methods utilize the off-the-shelf pose or parsing networks as pseudo labels, which are prone to error. We propose a novel Occlusion Correction Network (OCNet) that corrects features through relational-weight learning and obtains diverse and representative features without using external networks.
  arXiv  Detail & Related papers  (2022-12-09T07:48:47Z)
• Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification [21.006680330530852]
  We propose a feature pruning and consolidation (FPC) framework to circumvent explicit human structure parsing. The framework mainly consists of a sparse encoder, a multi-view feature mathcing module, and a feature consolidation decoder. Our method outperforms state-of-the-art results by at least 8.6% mAP and 6.0% Rank-1 accuracy on the challenging Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2022-11-27T06:18:40Z)
• Quality-aware Part Models for Occluded Person Re-identification [77.24920810798505]
  Occlusion poses a major challenge for person re-identification (ReID) Existing approaches typically rely on outside tools to infer visible body parts, which may be suboptimal in terms of both computational efficiency and ReID accuracy. We propose a novel method named Quality-aware Part Models (QPM) for occlusion-robust ReID.
  arXiv  Detail & Related papers  (2022-01-01T03:51:09Z)
• Self-Supervised Scene De-occlusion [186.89979151728636]
  This paper investigates the problem of scene de-occlusion, which aims to recover the underlying occlusion ordering and complete the invisible parts of occluded objects. We make the first attempt to address the problem through a novel and unified framework that recovers hidden scene structures without ordering and amodal annotations as supervisions. Based on PCNet-M and PCNet-C, we devise a novel inference scheme to accomplish scene de-occlusion, via progressive ordering recovery, amodal completion and content completion.
  arXiv  Detail & Related papers  (2020-04-06T16:31:11Z)
• Pose-guided Visible Part Matching for Occluded Person ReID [80.81748252960843]
  We propose a Pose-guided Visible Part Matching (PVPM) method that jointly learns the discriminative features with pose-guided attention and self-mines the part visibility. Experimental results on three reported occluded benchmarks show that the proposed method achieves competitive performance to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-04-01T04:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.