Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles

• URL: http://arxiv.org/abs/2210.06529v1
• Date: Wed, 12 Oct 2022 18:54:57 GMT
• Title: Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles
• Authors: Anjith George, Amir Mohammadi and Sebastien Marcel
• Abstract summary: We propose a surprisingly simple, yet, very effective method for matching face images across different sensing modalities. The proposed approach is architecture agnostic, meaning they can be added to any pre-trained models. The source code and protocols will be made available publicly.
• Score: 9.419177623349947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Heterogeneous Face Recognition (HFR) refers to matching face images captured in different domains, such as thermal to visible images (VIS), sketches to visible images, near-infrared to visible, and so on. This is particularly useful in matching visible spectrum images to images captured from other modalities. Though highly useful, HFR is challenging because of the domain gap between the source and target domain. Often, large-scale paired heterogeneous face image datasets are absent, preventing training models specifically for the heterogeneous task. In this work, we propose a surprisingly simple, yet, very effective method for matching face images across different sensing modalities. The core idea of the proposed approach is to add a novel neural network block called Prepended Domain Transformer (PDT) in front of a pre-trained face recognition (FR) model to address the domain gap. Retraining this new block with few paired samples in a contrastive learning setup was enough to achieve state-of-the-art performance in many HFR benchmarks. The PDT blocks can be retrained for several source-target combinations using the proposed general framework. The proposed approach is architecture agnostic, meaning they can be added to any pre-trained FR models. Further, the approach is modular and the new block can be trained with a minimal set of paired samples, making it much easier for practical deployment. The source code and protocols will be made available publicly.

Related papers

• MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection [64.29452783056253]
  The rapid development of photo-realistic face generation methods has raised significant concerns in society and academia. Although existing approaches mainly capture face forgery patterns using image modality, other modalities like fine-grained noises and texts are not fully explored. We propose a novel multi-modal fine-grained CLIP (MFCLIP) model, which mines comprehensive and fine-grained forgery traces across image-noise modalities.
  arXiv  Detail & Related papers  (2024-09-15T13:08:59Z)
• Heterogeneous Face Recognition Using Domain Invariant Units [4.910937238451485]
  We leverage a pretrained face recognition model as a teacher network to learn domaininvariant network layers called Domain-Invariant Units (DIU) The proposed DIU can be trained effectively even with a limited amount of paired training data, in a contrastive distillation framework. This proposed approach has the potential to enhance pretrained models, making them more adaptable to a wider range of variations in data.
  arXiv  Detail & Related papers  (2024-04-22T16:58:37Z)
• From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition [4.910937238451485]
  We present a new Conditional Adaptive Instance Modulation (CAIM) module that seamlessly fits into existing Face Recognition networks. The CAIM block modulates intermediate feature maps, efficiently adapting to the style of the source modality and bridging the domain gap. We extensively evaluate the proposed approach on various challenging HFR benchmarks, showing that it outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-04-22T15:00:51Z)
• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• Rethinking the Domain Gap in Near-infrared Face Recognition [65.7871950460781]
  Heterogeneous face recognition (HFR) involves the intricate task of matching face images across the visual domains of visible (VIS) and near-infrared (NIR) Much of the existing literature on HFR identifies the domain gap as a primary challenge and directs efforts towards bridging it at either the input or feature level. We observe that large neural networks, unlike their smaller counterparts, when pre-trained on large scale homogeneous VIS data, demonstrate exceptional zero-shot performance in HFR.
  arXiv  Detail & Related papers  (2023-12-01T14:43:28Z)
• Bridging the Gap: Heterogeneous Face Recognition with Conditional Adaptive Instance Modulation [7.665392786787577]
  We introduce a novel Conditional Adaptive Instance Modulation (CAIM) module that can be integrated into pre-trained Face Recognition networks. The CAIM block modulates intermediate feature maps, to adapt the style of the target modality effectively bridging the domain gap. Our proposed method allows for end-to-end training with a minimal number of paired samples.
  arXiv  Detail & Related papers  (2023-07-13T19:17:04Z)
• Hierarchical Forgery Classifier On Multi-modality Face Forgery Clues [61.37306431455152]
  We propose a novel Hierarchical Forgery for Multi-modality Face Forgery Detection (HFC-MFFD) The HFC-MFFD learns robust patches-based hybrid representation to enhance forgery authentication in multiple-modality scenarios. The specific hierarchical face forgery is proposed to alleviate the class imbalance problem and further boost detection performance.
  arXiv  Detail & Related papers  (2022-12-30T10:54:29Z)
• Dynamic Prototype Mask for Occluded Person Re-Identification [88.7782299372656]
  Existing methods mainly address this issue by employing body clues provided by an extra network to distinguish the visible part. We propose a novel Dynamic Prototype Mask (DPM) based on two self-evident prior knowledge. Under this condition, the occluded representation could be well aligned in a selected subspace spontaneously.
  arXiv  Detail & Related papers  (2022-07-19T03:31:13Z)
• Multi-Margin based Decorrelation Learning for Heterogeneous Face Recognition [90.26023388850771]
  This paper presents a deep neural network approach to extract decorrelation representations in a hyperspherical space for cross-domain face images. The proposed framework can be divided into two components: heterogeneous representation network and decorrelation representation learning. Experimental results on two challenging heterogeneous face databases show that our approach achieves superior performance on both verification and recognition tasks.
  arXiv  Detail & Related papers  (2020-05-25T07:01:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.