DLCR: A Generative Data Expansion Framework via Diffusion for Clothes-Changing Person Re-ID

• URL: http://arxiv.org/abs/2411.07205v1
• Date: Mon, 11 Nov 2024 18:28:33 GMT
• Title: DLCR: A Generative Data Expansion Framework via Diffusion for Clothes-Changing Person Re-ID
• Authors: Nyle Siddiqui, Florinel Alin Croitoru, Gaurav Kumar Nayak, Radu Tudor Ionescu, Mubarak Shah,
• Abstract summary: We propose a novel data expansion framework to generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets. We obtain a large top-1 accuracy improvement of $11.3%$ by training CAL, a previous state of the art (SOTA) method, with DLCR-generated data.
• Score: 69.70281727931048
• License:
• Abstract: With the recent exhibited strength of generative diffusion models, an open research question is \textit{if images generated by these models can be used to learn better visual representations}. While this generative data expansion may suffice for easier visual tasks, we explore its efficacy on a more difficult discriminative task: clothes-changing person re-identification (CC-ReID). CC-ReID aims to match people appearing in non-overlapping cameras, even when they change their clothes across cameras. Not only are current CC-ReID models constrained by the limited diversity of clothing in current CC-ReID datasets, but generating additional data that retains important personal features for accurate identification is a current challenge. To address this issue we propose DLCR, a novel data expansion framework that leverages pre-trained diffusion and large language models (LLMs) to accurately generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets (PRCC, CCVID, LaST, VC-Clothes, and LTCC) and \textbf{increase their clothing diversity by \boldmath{$10$}x, totaling over \boldmath{$2.1$}M images generated}. DLCR employs diffusion-based text-guided inpainting, conditioned on clothing prompts constructed using LLMs, to generate synthetic data that only modifies a subject's clothes while preserving their personally identifiable features. With this massive increase in data, we introduce two novel strategies - progressive learning and test-time prediction refinement - that respectively reduce training time and further boosts CC-ReID performance. On the PRCC dataset, we obtain a large top-1 accuracy improvement of $11.3\%$ by training CAL, a previous state of the art (SOTA) method, with DLCR-generated data. We publicly release our code and generated data for each dataset here: \url{https://github.com/CroitoruAlin/dlcr}.

Related papers

• CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models [6.892813084970311]
  Cloth-changing person re-identification (CC-ReID) is a critical and challenging research topic in computer vision. Due to the high cost of constructing CC-ReID data, the existing data-driven models are hard to train efficiently on limited data. We propose a low-cost and efficient pipeline for generating controllable and high-quality synthetic data.
  arXiv  Detail & Related papers  (2024-10-17T14:04:02Z)
• Synthesizing Efficient Data with Diffusion Models for Person Re-Identification Pre-Training [51.87027943520492]
  We present a novel paradigm Diffusion-ReID to efficiently augment and generate diverse images based on known identities. Benefiting from our proposed paradigm, we first create a new large-scale person Re-ID dataset Diff-Person, which consists of over 777K images from 5,183 identities.
  arXiv  Detail & Related papers  (2024-06-10T06:26:03Z)
• CCPA: Long-term Person Re-Identification via Contrastive Clothing and Pose Augmentation [2.1756081703276]
  Long-term Person Re-Identification aims at matching an individual across cameras after a long period of time. We propose CCPA: Contrastive Clothing and Pose Augmentation framework for LRe-ID.
  arXiv  Detail & Related papers  (2024-02-22T11:16:34Z)
• Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification [78.52704557647438]
  We propose a novel FIne-grained Representation and Recomposition (FIRe$2$) framework to tackle both limitations without any auxiliary annotation or data. Experiments demonstrate that FIRe$2$ can achieve state-of-the-art performance on five widely-used cloth-changing person Re-ID benchmarks.
  arXiv  Detail & Related papers  (2023-08-21T12:59:48Z)
• GEFF: Improving Any Clothes-Changing Person ReID Model using Gallery Enrichment with Face Features [11.189236254478057]
  In Clothes-Changing Re-Identification (CC-ReID) problem, given a query sample of a person, the goal is to determine the correct identity based on a labeled gallery in which the person appears in different clothes. Several models tackle this challenge by extracting clothes-independent features. As clothing-related features are often dominant features in the data, we propose a new process we call Gallery Enrichment.
  arXiv  Detail & Related papers  (2022-11-24T21:41:52Z)
• Clothes-Changing Person Re-identification with RGB Modality Only [102.44387094119165]
  We propose a Clothes-based Adrial Loss (CAL) to mine clothes-irrelevant features from the original RGB images. Videos contain richer appearance and additional temporal information, which can be used to model propertemporal patterns.
  arXiv  Detail & Related papers  (2022-04-14T11:38:28Z)
• Unsupervised Pre-training for Person Re-identification [90.98552221699508]
  We present a large scale unlabeled person re-identification (Re-ID) dataset "LUPerson" We make the first attempt of performing unsupervised pre-training for improving the generalization ability of the learned person Re-ID feature representation.
  arXiv  Detail & Related papers  (2020-12-07T14:48:26Z)
• Long-Term Cloth-Changing Person Re-identification [154.57752691285046]
  Person re-identification (Re-ID) aims to match a target person across camera views at different locations and times. Existing Re-ID studies focus on the short-term cloth-consistent setting, under which a person re-appears in different camera views with the same outfit. In this work, we focus on a much more difficult yet practical setting where person matching is conducted over long-duration, e.g., over days and months.
  arXiv  Detail & Related papers  (2020-05-26T11:27:21Z)
• Learning Shape Representations for Clothing Variations in Person Re-Identification [34.559050607889816]
  Person re-identification (re-ID) aims to recognize instances of the same person contained in multiple images taken across different cameras. We propose a novel representation learning model which is able to generate a body shape feature representation without being affected by clothing color or patterns. Case-Net learns a representation of identity that depends only on body shape via adversarial learning and feature disentanglement.
  arXiv  Detail & Related papers  (2020-03-16T17:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.