From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition
- URL: http://arxiv.org/abs/2404.14247v1
- Date: Mon, 22 Apr 2024 15:00:51 GMT
- Title: From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition
- Authors: Anjith George, Sebastien Marcel,
- Abstract summary: We present a new Conditional Adaptive Instance Modulation (CAIM) module that seamlessly fits into existing Face Recognition networks.
The CAIM block modulates intermediate feature maps, efficiently adapting to the style of the source modality and bridging the domain gap.
We extensively evaluate the proposed approach on various challenging HFR benchmarks, showing that it outperforms state-of-the-art methods.
- Score: 4.910937238451485
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Heterogeneous Face Recognition (HFR) focuses on matching faces from different domains, for instance, thermal to visible images, making Face Recognition (FR) systems more versatile for challenging scenarios. However, the domain gap between these domains and the limited large-scale datasets in the target HFR modalities make it challenging to develop robust HFR models from scratch. In our work, we view different modalities as distinct styles and propose a method to modulate feature maps of the target modality to address the domain gap. We present a new Conditional Adaptive Instance Modulation (CAIM ) module that seamlessly fits into existing FR networks, turning them into HFR-ready systems. The CAIM block modulates intermediate feature maps, efficiently adapting to the style of the source modality and bridging the domain gap. Our method enables end-to-end training using a small set of paired samples. We extensively evaluate the proposed approach on various challenging HFR benchmarks, showing that it outperforms state-of-the-art methods. The source code and protocols for reproducing the findings will be made publicly available
Related papers
- Exploiting Aggregation and Segregation of Representations for Domain Adaptive Human Pose Estimation [50.31351006532924]
Human pose estimation (HPE) has received increasing attention recently due to its wide application in motion analysis, virtual reality, healthcare, etc.
It suffers from the lack of labeled diverse real-world datasets due to the time- and labor-intensive annotation.
We introduce a novel framework that capitalizes on both representation aggregation and segregation for domain adaptive human pose estimation.
arXiv Detail & Related papers (2024-12-29T17:59:45Z) - Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification [57.945437355714155]
Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions.
Existing approaches focus on single-source domain generalization to unseen target domains.
We propose a novel multi-source collaborative domain generalization framework (MS-CDG) based on homogeneity and heterogeneity characteristics of multi-source remote sensing data.
arXiv Detail & Related papers (2024-12-05T06:15:08Z) - Modality Agnostic Heterogeneous Face Recognition with Switch Style Modulators [4.910937238451485]
We introduce a novel framework designed to train a modality-agnostic HFR method capable of handling multiple modalities during inference.
We achieve this by implementing a computationally efficient automatic routing mechanism called Switch Style Modulation Blocks (SSMB)
Our proposed SSMB can be trained end-to-end and seamlessly integrated into pre-trained face recognition models, transforming them into modality-agnostic HFR models.
arXiv Detail & Related papers (2024-07-11T16:21:48Z) - Heterogeneous Face Recognition Using Domain Invariant Units [4.910937238451485]
We leverage a pretrained face recognition model as a teacher network to learn domaininvariant network layers called Domain-Invariant Units (DIU)
The proposed DIU can be trained effectively even with a limited amount of paired training data, in a contrastive distillation framework.
This proposed approach has the potential to enhance pretrained models, making them more adaptable to a wider range of variations in data.
arXiv Detail & Related papers (2024-04-22T16:58:37Z) - DARNet: Bridging Domain Gaps in Cross-Domain Few-Shot Segmentation with
Dynamic Adaptation [20.979759016826378]
Few-shot segmentation (FSS) aims to segment novel classes in a query image by using only a small number of supporting images from base classes.
In cross-domain FSS, leveraging features from label-rich domains for resource-constrained domains poses challenges due to domain discrepancies.
This work presents a Dynamically Adaptive Refine (DARNet) method that aims to balance generalization and specificity for CD-FSS.
arXiv Detail & Related papers (2023-12-08T03:03:22Z) - Rethinking the Domain Gap in Near-infrared Face Recognition [65.7871950460781]
Heterogeneous face recognition (HFR) involves the intricate task of matching face images across the visual domains of visible (VIS) and near-infrared (NIR)
Much of the existing literature on HFR identifies the domain gap as a primary challenge and directs efforts towards bridging it at either the input or feature level.
We observe that large neural networks, unlike their smaller counterparts, when pre-trained on large scale homogeneous VIS data, demonstrate exceptional zero-shot performance in HFR.
arXiv Detail & Related papers (2023-12-01T14:43:28Z) - Bridging the Gap: Heterogeneous Face Recognition with Conditional
Adaptive Instance Modulation [7.665392786787577]
We introduce a novel Conditional Adaptive Instance Modulation (CAIM) module that can be integrated into pre-trained Face Recognition networks.
The CAIM block modulates intermediate feature maps, to adapt the style of the target modality effectively bridging the domain gap.
Our proposed method allows for end-to-end training with a minimal number of paired samples.
arXiv Detail & Related papers (2023-07-13T19:17:04Z) - Prepended Domain Transformer: Heterogeneous Face Recognition without
Bells and Whistles [9.419177623349947]
We propose a surprisingly simple, yet, very effective method for matching face images across different sensing modalities.
The proposed approach is architecture agnostic, meaning they can be added to any pre-trained models.
The source code and protocols will be made available publicly.
arXiv Detail & Related papers (2022-10-12T18:54:57Z) - Deep face recognition with clustering based domain adaptation [57.29464116557734]
We propose a new clustering-based domain adaptation method designed for face recognition task in which the source and target domain do not share any classes.
Our method effectively learns the discriminative target feature by aligning the feature domain globally, and, at the meantime, distinguishing the target clusters locally.
arXiv Detail & Related papers (2022-05-27T12:29:11Z) - Inter-class Discrepancy Alignment for Face Recognition [55.578063356210144]
We propose a unified framework calledInter-class DiscrepancyAlignment(IDA)
IDA-DAO is used to align the similarity scores considering the discrepancy between the images and its neighbors.
IDA-SSE can provide convincing inter-class neighbors by introducing virtual candidate images generated with GAN.
arXiv Detail & Related papers (2021-03-02T08:20:08Z) - Learning Meta Face Recognition in Unseen Domains [74.69681594452125]
We propose a novel face recognition method via meta-learning named Meta Face Recognition (MFR)
MFR synthesizes the source/target domain shift with a meta-optimization objective.
We propose two benchmarks for generalized face recognition evaluation.
arXiv Detail & Related papers (2020-03-17T14:10:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.