From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition

• URL: http://arxiv.org/abs/2404.14247v1
• Date: Mon, 22 Apr 2024 15:00:51 GMT
• Title: From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition
• Authors: Anjith George, Sebastien Marcel,
• Abstract summary: We present a new Conditional Adaptive Instance Modulation (CAIM) module that seamlessly fits into existing Face Recognition networks. The CAIM block modulates intermediate feature maps, efficiently adapting to the style of the source modality and bridging the domain gap. We extensively evaluate the proposed approach on various challenging HFR benchmarks, showing that it outperforms state-of-the-art methods.
• Score: 4.910937238451485
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Heterogeneous Face Recognition (HFR) focuses on matching faces from different domains, for instance, thermal to visible images, making Face Recognition (FR) systems more versatile for challenging scenarios. However, the domain gap between these domains and the limited large-scale datasets in the target HFR modalities make it challenging to develop robust HFR models from scratch. In our work, we view different modalities as distinct styles and propose a method to modulate feature maps of the target modality to address the domain gap. We present a new Conditional Adaptive Instance Modulation (CAIM ) module that seamlessly fits into existing FR networks, turning them into HFR-ready systems. The CAIM block modulates intermediate feature maps, efficiently adapting to the style of the source modality and bridging the domain gap. Our method enables end-to-end training using a small set of paired samples. We extensively evaluate the proposed approach on various challenging HFR benchmarks, showing that it outperforms state-of-the-art methods. The source code and protocols for reproducing the findings will be made publicly available

Related papers

• Modality Agnostic Heterogeneous Face Recognition with Switch Style Modulators [4.910937238451485]
  We introduce a novel framework designed to train a modality-agnostic HFR method capable of handling multiple modalities during inference. We achieve this by implementing a computationally efficient automatic routing mechanism called Switch Style Modulation Blocks (SSMB) Our proposed SSMB can be trained end-to-end and seamlessly integrated into pre-trained face recognition models, transforming them into modality-agnostic HFR models.
  arXiv  Detail & Related papers  (2024-07-11T16:21:48Z)
• Heterogeneous Face Recognition Using Domain Invariant Units [4.910937238451485]
  We leverage a pretrained face recognition model as a teacher network to learn domaininvariant network layers called Domain-Invariant Units (DIU) The proposed DIU can be trained effectively even with a limited amount of paired training data, in a contrastive distillation framework. This proposed approach has the potential to enhance pretrained models, making them more adaptable to a wider range of variations in data.
  arXiv  Detail & Related papers  (2024-04-22T16:58:37Z)
• DARNet: Bridging Domain Gaps in Cross-Domain Few-Shot Segmentation with Dynamic Adaptation [20.979759016826378]
  Few-shot segmentation (FSS) aims to segment novel classes in a query image by using only a small number of supporting images from base classes. In cross-domain FSS, leveraging features from label-rich domains for resource-constrained domains poses challenges due to domain discrepancies. This work presents a Dynamically Adaptive Refine (DARNet) method that aims to balance generalization and specificity for CD-FSS.
  arXiv  Detail & Related papers  (2023-12-08T03:03:22Z)
• Rethinking the Domain Gap in Near-infrared Face Recognition [65.7871950460781]
  Heterogeneous face recognition (HFR) involves the intricate task of matching face images across the visual domains of visible (VIS) and near-infrared (NIR) Much of the existing literature on HFR identifies the domain gap as a primary challenge and directs efforts towards bridging it at either the input or feature level. We observe that large neural networks, unlike their smaller counterparts, when pre-trained on large scale homogeneous VIS data, demonstrate exceptional zero-shot performance in HFR.
  arXiv  Detail & Related papers  (2023-12-01T14:43:28Z)
• Bridging the Gap: Heterogeneous Face Recognition with Conditional Adaptive Instance Modulation [7.665392786787577]
  We introduce a novel Conditional Adaptive Instance Modulation (CAIM) module that can be integrated into pre-trained Face Recognition networks. The CAIM block modulates intermediate feature maps, to adapt the style of the target modality effectively bridging the domain gap. Our proposed method allows for end-to-end training with a minimal number of paired samples.
  arXiv  Detail & Related papers  (2023-07-13T19:17:04Z)
• Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles [9.419177623349947]
  We propose a surprisingly simple, yet, very effective method for matching face images across different sensing modalities. The proposed approach is architecture agnostic, meaning they can be added to any pre-trained models. The source code and protocols will be made available publicly.
  arXiv  Detail & Related papers  (2022-10-12T18:54:57Z)
• Deep face recognition with clustering based domain adaptation [57.29464116557734]
  We propose a new clustering-based domain adaptation method designed for face recognition task in which the source and target domain do not share any classes. Our method effectively learns the discriminative target feature by aligning the feature domain globally, and, at the meantime, distinguishing the target clusters locally.
  arXiv  Detail & Related papers  (2022-05-27T12:29:11Z)
• Inter-class Discrepancy Alignment for Face Recognition [55.578063356210144]
  We propose a unified framework calledInter-class DiscrepancyAlignment(IDA) IDA-DAO is used to align the similarity scores considering the discrepancy between the images and its neighbors. IDA-SSE can provide convincing inter-class neighbors by introducing virtual candidate images generated with GAN.
  arXiv  Detail & Related papers  (2021-03-02T08:20:08Z)
• Domain Conditioned Adaptation Network [90.63261870610211]
  We propose a Domain Conditioned Adaptation Network (DCAN) to excite distinct convolutional channels with a domain conditioned channel attention mechanism. This is the first work to explore the domain-wise convolutional channel activation for deep DA networks.
  arXiv  Detail & Related papers  (2020-05-14T04:23:24Z)
• Learning Meta Face Recognition in Unseen Domains [74.69681594452125]
  We propose a novel face recognition method via meta-learning named Meta Face Recognition (MFR) MFR synthesizes the source/target domain shift with a meta-optimization objective. We propose two benchmarks for generalized face recognition evaluation.
  arXiv  Detail & Related papers  (2020-03-17T14:10:30Z)
• Unsupervised Domain Adaptation in Person re-ID via k-Reciprocal Clustering and Large-Scale Heterogeneous Environment Synthesis [76.46004354572956]
  We introduce an unsupervised domain adaptation approach for person re-identification. Experimental results show that the proposed ktCUDA and SHRED approach achieves an average improvement of +5.7 mAP in re-identification performance.
  arXiv  Detail & Related papers  (2020-01-14T17:43:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.