Bidirectional Multi-Step Domain Generalization for Visible-Infrared Person Re-Identification

• URL: http://arxiv.org/abs/2403.10782v1
• Date: Sat, 16 Mar 2024 03:03:27 GMT
• Title: Bidirectional Multi-Step Domain Generalization for Visible-Infrared Person Re-Identification
• Authors: Mahdi Alehdaghi, Pourya Shamsolmoali, Rafael M. O. Cruz, Eric Granger,
• Abstract summary: A key challenge in visible-infrared person re-identification (V-I ReID) is training a backbone model capable of effectively addressing the significant discrepancies across modalities. This paper introduces the Bidirectional Multi-step Domain Generalization, a novel approach for unifying feature representations across diverse modalities. Experiments conducted on challenging V-I ReID datasets indicate that our BMDG approach outperforms state-of-the-art part-based models or methods that generate an intermediate domain from V-I person ReID.
• Score: 12.14946364107671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A key challenge in visible-infrared person re-identification (V-I ReID) is training a backbone model capable of effectively addressing the significant discrepancies across modalities. State-of-the-art methods that generate a single intermediate bridging domain are often less effective, as this generated domain may not adequately capture sufficient common discriminant information. This paper introduces the Bidirectional Multi-step Domain Generalization (BMDG), a novel approach for unifying feature representations across diverse modalities. BMDG creates multiple virtual intermediate domains by finding and aligning body part features extracted from both I and V modalities. Indeed, BMDG aims to reduce the modality gaps in two steps. First, it aligns modalities in feature space by learning shared and modality-invariant body part prototypes from V and I images. Then, it generalizes the feature representation by applying bidirectional multi-step learning, which progressively refines feature representations in each step and incorporates more prototypes from both modalities. In particular, our method minimizes the cross-modal gap by identifying and aligning shared prototypes that capture key discriminative features across modalities, then uses multiple bridging steps based on this information to enhance the feature representation. Experiments conducted on challenging V-I ReID datasets indicate that our BMDG approach outperforms state-of-the-art part-based models or methods that generate an intermediate domain from V-I person ReID.

Related papers

• Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification [11.664820595258988]
  Primary challenges in visible-infrared person re-identification arise from the differences between visible (vis) and infrared (ir) images. Existing methods often rely on horizontal partitioning to align part-level features, which can introduce inaccuracies. We propose a novel Prototype-Driven Multi-feature generation framework (PDM) aimed at mitigating cross-modal discrepancies.
  arXiv  Detail & Related papers  (2024-09-09T14:12:23Z)
• HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization [69.33162366130887]
  Domain Generalization (DG) endeavors to create machine learning models that excel in unseen scenarios by learning invariant features. We introduce a novel method designed to supplement the model with domain-level and task-specific characteristics. This approach aims to guide the model in more effectively separating invariant features from specific characteristics, thereby boosting the generalization.
  arXiv  Detail & Related papers  (2024-01-18T04:23:21Z)
• Modality Unifying Network for Visible-Infrared Person Re-Identification [24.186989535051623]
  Visible-infrared person re-identification (VI-ReID) is a challenging task due to large cross-modality discrepancies and intra-class variations. Existing methods mainly focus on learning modality-shared representations by embedding different modalities into the same feature space. We propose a novel Modality Unifying Network (MUN) to explore a robust auxiliary modality for VI-ReID.
  arXiv  Detail & Related papers  (2023-09-12T14:22:22Z)
• Adaptive Generation of Privileged Intermediate Information for Visible-Infrared Person Re-Identification [11.93952924941977]
  This paper introduces the Adaptive Generation of Privileged Intermediate Information training approach. AGPI2 is introduced to adapt and generate a virtual domain that bridges discriminant information between the V and I modalities. Experimental results conducted on challenging V-I ReID indicate that AGPI2 increases matching accuracy without extra computational resources.
  arXiv  Detail & Related papers  (2023-07-06T18:08:36Z)
• Causality-based Dual-Contrastive Learning Framework for Domain Generalization [16.81075442901155]
  Domain Generalization (DG) is essentially a sub-branch of out-of-distribution generalization. In this paper, we propose a Dual-Contrastive Learning (DCL) module on feature and prototype contrast. We also introduce a Similarity-based Hard-pair Mining (SHM) strategy to leverage information on diversity shift.
  arXiv  Detail & Related papers  (2023-01-22T13:07:24Z)
• Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities [76.08541852988536]
  We propose to use invariant features for a missing modality imagination network (IF-MMIN) We show that the proposed model outperforms all baselines and invariantly improves the overall emotion recognition performance under uncertain missing-modality conditions.
  arXiv  Detail & Related papers  (2022-10-27T12:16:25Z)
• A Novel Mix-normalization Method for Generalizable Multi-source Person Re-identification [49.548815417844786]
  Person re-identification (Re-ID) has achieved great success in the supervised scenario. It is difficult to directly transfer the supervised model to arbitrary unseen domains due to the model overfitting to the seen source domains. We propose MixNorm, which consists of domain-aware mix-normalization (DMN) and domain-ware center regularization (DCR)
  arXiv  Detail & Related papers  (2022-01-24T18:09:38Z)
• MSO: Multi-Feature Space Joint Optimization Network for RGB-Infrared Person Re-Identification [35.97494894205023]
  RGB-infrared cross-modality person re-identification (ReID) task aims to recognize the images of the same identity between the visible modality and the infrared modality. Existing methods mainly use a two-stream architecture to eliminate the discrepancy between the two modalities in the final common feature space. We present a novel multi-feature space joint optimization (MSO) network, which can learn modality-sharable features in both the single-modality space and the common space.
  arXiv  Detail & Related papers  (2021-10-21T16:45:23Z)
• Learning Domain Invariant Representations for Generalizable Person Re-Identification [71.35292121563491]
  Generalizable person Re-Identification (ReID) has attracted growing attention in recent computer vision community. We introduce causality into person ReID and propose a novel generalizable framework, named Domain Invariant Representations for generalizable person Re-Identification (DIR-ReID)
  arXiv  Detail & Related papers  (2021-03-29T18:59:48Z)
• Multi-Domain Adversarial Feature Generalization for Person Re-Identification [52.835955258959785]
  We propose a multi-dataset feature generalization network (MMFA-AAE) It is capable of learning a universal domain-invariant feature representation from multiple labeled datasets and generalizing it to unseen' camera systems. It also surpasses many state-of-the-art supervised methods and unsupervised domain adaptation methods by a large margin.
  arXiv  Detail & Related papers  (2020-11-25T08:03:15Z)
• Dual Distribution Alignment Network for Generalizable Person Re-Identification [174.36157174951603]
  Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID) We present a Dual Distribution Alignment Network (DDAN) which handles this challenge by selectively aligning distributions of multiple source domains. We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark.
  arXiv  Detail & Related papers  (2020-07-27T00:08:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.