Multi-Domain Biometric Recognition using Body Embeddings

• URL: http://arxiv.org/abs/2503.10931v1
• Date: Thu, 13 Mar 2025 22:38:18 GMT
• Title: Multi-Domain Biometric Recognition using Body Embeddings
• Authors: Anirudh Nanduri, Siyuan Huang, Rama Chellappa,
• Abstract summary: We show that body embeddings perform better than face embeddings in medium-wave infrared (MWIR) and long-wave infrared (LWIR) domains.<n>We leverage a vision transformer architecture to establish benchmark results on the IJB-MDF dataset.<n>We also show that finetuning a body model, pretrained exclusively on VIS data, with a simple combination of cross-entropy and triplet losses achieves state-of-the-art mAP scores.
• Score: 51.36007967653781
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biometric recognition becomes increasingly challenging as we move away from the visible spectrum to infrared imagery, where domain discrepancies significantly impact identification performance. In this paper, we show that body embeddings perform better than face embeddings for cross-spectral person identification in medium-wave infrared (MWIR) and long-wave infrared (LWIR) domains. Due to the lack of multi-domain datasets, previous research on cross-spectral body identification - also known as Visible-Infrared Person Re-Identification (VI-ReID) - has primarily focused on individual infrared bands, such as near-infrared (NIR) or LWIR, separately. We address the multi-domain body recognition problem using the IARPA Janus Benchmark Multi-Domain Face (IJB-MDF) dataset, which enables matching of short-wave infrared (SWIR), MWIR, and LWIR images against RGB (VIS) images. We leverage a vision transformer architecture to establish benchmark results on the IJB-MDF dataset and, through extensive experiments, provide valuable insights into the interrelation of infrared domains, the adaptability of VIS-pretrained models, the role of local semantic features in body-embeddings, and effective training strategies for small datasets. Additionally, we show that finetuning a body model, pretrained exclusively on VIS data, with a simple combination of cross-entropy and triplet losses achieves state-of-the-art mAP scores on the LLCM dataset.

Related papers

• Cross-Spectral Attention for Unsupervised RGB-IR Face Verification and Person Re-identification [1.6768151308423365]
  Cross-spectral biometrics, such as matching imagery of faces or persons from visible (RGB) and infrared (IR) bands, have rapidly advanced over the last decade.<n>We propose a novel unsupervised cross-spectral framework that combines (1) a new pseudo triplet loss with cross-spectral voting, (2) a new cross-spectral attention network leveraging multiple subspaces, and (3) structured sparsity to perform more discriminative cross-spectral clustering.
  arXiv  Detail & Related papers  (2024-11-28T15:38:15Z)
• Frequency Domain Nuances Mining for Visible-Infrared Person Re-identification [75.87443138635432]
  Existing methods mainly exploit the spatial information while ignoring the discriminative frequency information. We propose a novel Frequency Domain Nuances Mining (FDNM) method to explore the cross-modality frequency domain information. Our method outperforms the second-best method by 5.2% in Rank-1 accuracy and 5.8% in mAP on the SYSU-MM01 dataset.
  arXiv  Detail & Related papers  (2024-01-04T09:19:54Z)
• Diverse Embedding Expansion Network and Low-Light Cross-Modality Benchmark for Visible-Infrared Person Re-identification [26.71900654115498]
  We propose a novel augmentation network in the embedding space, called diverse embedding expansion network (DEEN) The proposed DEEN can effectively generate diverse embeddings to learn the informative feature representations. We provide a low-light cross-modality (LLCM) dataset, which contains 46,767 bounding boxes of 1,064 identities captured by 9 RGB/IR cameras.
  arXiv  Detail & Related papers  (2023-03-25T14:24:56Z)
• Exploring Invariant Representation for Visible-Infrared Person Re-Identification [77.06940947765406]
  Cross-spectral person re-identification, which aims to associate identities to pedestrians across different spectra, faces a main challenge of the modality discrepancy. In this paper, we address the problem from both image-level and feature-level in an end-to-end hybrid learning framework named robust feature mining network (RFM) Experiment results on two standard cross-spectral person re-identification datasets, RegDB and SYSU-MM01, have demonstrated state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-02-02T05:24:50Z)
• Physically-Based Face Rendering for NIR-VIS Face Recognition [165.54414962403555]
  Near infrared (NIR) to Visible (VIS) face matching is challenging due to the significant domain gaps. We propose a novel method for paired NIR-VIS facial image generation. To facilitate the identity feature learning, we propose an IDentity-based Maximum Mean Discrepancy (ID-MMD) loss.
  arXiv  Detail & Related papers  (2022-11-11T18:48:16Z)
• Visible-Infrared Person Re-Identification Using Privileged Intermediate Information [10.816003787786766]
  Cross-modal person re-identification (ReID) is challenging due to the large domain shift in data distributions between RGB and IR modalities. This paper introduces a novel approach for a creating intermediate virtual domain that acts as bridges between the two main domains. We devised a new method to generate images between visible and infrared domains that provide additional information to train a deep ReID model.
  arXiv  Detail & Related papers  (2022-09-19T21:08:14Z)
• Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection [2.478658210785]
  We exploit both visual and thermal perception units for robust object detection purposes. In this paper, we exploit both visual and thermal perception units for robust object detection purposes.
  arXiv  Detail & Related papers  (2022-06-08T15:02:58Z)
• Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration [59.02821429555375]
  We present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion. To better fuse the registered infrared images and visible images, we present a feature Interaction Fusion Module (IFM)
  arXiv  Detail & Related papers  (2022-05-24T07:51:57Z)
• Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds [155.388487263872]
  We propose a new infrared small-dim target detection method with the transformer. We adopt the self-attention mechanism of the transformer to learn the interaction information of image features in a larger range. We also design a feature enhancement module to learn more features of small-dim targets.
  arXiv  Detail & Related papers  (2021-09-29T12:23:41Z)
• Cross-Spectral Iris Matching Using Conditional Coupled GAN [22.615156512223766]
  Cross-spectral iris recognition is emerging as a promising biometric approach to authenticating the identity of individuals. matching iris images acquired at different spectral bands shows significant performance degradation when compared to single-band near-infrared (NIR) matching. We propose a conditional coupled generative adversarial network (CpGAN) architecture for cross-spectral iris recognition.
  arXiv  Detail & Related papers  (2020-10-09T19:13:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.