Multi-Modal Face Anti-Spoofing via Cross-Modal Feature Transitions

• URL: http://arxiv.org/abs/2507.05575v1
• Date: Tue, 08 Jul 2025 01:24:14 GMT
• Title: Multi-Modal Face Anti-Spoofing via Cross-Modal Feature Transitions
• Authors: Jun-Xiong Chong, Fang-Yu Hsu, Ming-Tsung Hsu, Yi-Ting Lin, Kai-Heng Chien, Chiou-Ting Hsu, Pei-Kai Huang,
• Abstract summary: Multi-modal face anti-spoofing (FAS) aims to detect genuine human presence by extracting liveness cues from multiple modalities.<n>We propose a novel Cross-modal Transition-guided Network (CTNet) to tackle the challenges in the multi-modal FAS task.
• Score: 2.3505792985856813
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal face anti-spoofing (FAS) aims to detect genuine human presence by extracting discriminative liveness cues from multiple modalities, such as RGB, infrared (IR), and depth images, to enhance the robustness of biometric authentication systems. However, because data from different modalities are typically captured by various camera sensors and under diverse environmental conditions, multi-modal FAS often exhibits significantly greater distribution discrepancies across training and testing domains compared to single-modal FAS. Furthermore, during the inference stage, multi-modal FAS confronts even greater challenges when one or more modalities are unavailable or inaccessible. In this paper, we propose a novel Cross-modal Transition-guided Network (CTNet) to tackle the challenges in the multi-modal FAS task. Our motivation stems from that, within a single modality, the visual differences between live faces are typically much smaller than those of spoof faces. Additionally, feature transitions across modalities are more consistent for the live class compared to those between live and spoof classes. Upon this insight, we first propose learning consistent cross-modal feature transitions among live samples to construct a generalized feature space. Next, we introduce learning the inconsistent cross-modal feature transitions between live and spoof samples to effectively detect out-of-distribution (OOD) attacks during inference. To further address the issue of missing modalities, we propose learning complementary infrared (IR) and depth features from the RGB modality as auxiliary modalities. Extensive experiments demonstrate that the proposed CTNet outperforms previous two-class multi-modal FAS methods across most protocols.

Related papers

• Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification [11.664820595258988]
  Primary challenges in visible-infrared person re-identification arise from the differences between visible (vis) and infrared (ir) images. Existing methods often rely on horizontal partitioning to align part-level features, which can introduce inaccuracies. We propose a novel Prototype-Driven Multi-feature generation framework (PDM) aimed at mitigating cross-modal discrepancies.
  arXiv  Detail & Related papers  (2024-09-09T14:12:23Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• Cross-Modality Perturbation Synergy Attack for Person Re-identification [66.48494594909123]
  Cross-modality person re-identification (ReID) systems are based on RGB images.<n>Main challenge in cross-modality ReID lies in effectively dealing with visual differences between different modalities.<n>Existing attack methods have primarily focused on the characteristics of the visible image modality.<n>This study proposes a universal perturbation attack specifically designed for cross-modality ReID.
  arXiv  Detail & Related papers  (2024-01-18T15:56:23Z)
• Modality Unifying Network for Visible-Infrared Person Re-Identification [24.186989535051623]
  Visible-infrared person re-identification (VI-ReID) is a challenging task due to large cross-modality discrepancies and intra-class variations. Existing methods mainly focus on learning modality-shared representations by embedding different modalities into the same feature space. We propose a novel Modality Unifying Network (MUN) to explore a robust auxiliary modality for VI-ReID.
  arXiv  Detail & Related papers  (2023-09-12T14:22:22Z)
• Visual Prompt Flexible-Modal Face Anti-Spoofing [23.58674017653937]
  multimodal face data collected from the real world is often imperfect due to missing modalities from various imaging sensors. We propose flexible-modal FAS, which learns the modal-relevant prompts to adapt the frozen pre-trained foundation model to downstream flexible-modal FAS task. experiments conducted on two multimodal FAS benchmark datasets demonstrate the effectiveness of our VP-FAS framework.
  arXiv  Detail & Related papers  (2023-07-26T05:06:41Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Hierarchical Forgery Classifier On Multi-modality Face Forgery Clues [61.37306431455152]
  We propose a novel Hierarchical Forgery for Multi-modality Face Forgery Detection (HFC-MFFD) The HFC-MFFD learns robust patches-based hybrid representation to enhance forgery authentication in multiple-modality scenarios. The specific hierarchical face forgery is proposed to alleviate the class imbalance problem and further boost detection performance.
  arXiv  Detail & Related papers  (2022-12-30T10:54:29Z)
• Flexible-Modal Face Anti-Spoofing: A Benchmark [66.18359076810549]
  Face anti-spoofing (FAS) plays a vital role in securing face recognition systems from presentation attacks. We establish the first flexible-modal FAS benchmark with the principle train one for all' We also investigate prevalent deep models and feature fusion strategies for flexible-modal FAS.
  arXiv  Detail & Related papers  (2022-02-16T16:55:39Z)
• Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction [125.18248926508045]
  We propose Channel-Exchanging-Network (CEN) which is self-adaptive, parameter-free, and more importantly, applicable for both multimodal fusion and multitask learning. CEN dynamically exchanges channels betweenworks of different modalities. For the application of dense image prediction, the validity of CEN is tested by four different scenarios.
  arXiv  Detail & Related papers  (2021-12-04T05:47:54Z)
• Multi-Scale Cascading Network with Compact Feature Learning for RGB-Infrared Person Re-Identification [35.55895776505113]
  Multi-Scale Part-Aware Cascading framework (MSPAC) is formulated by aggregating multi-scale fine-grained features from part to global. Cross-modality correlations can thus be efficiently explored on salient features for distinctive modality-invariant feature learning.
  arXiv  Detail & Related papers  (2020-12-12T15:39:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.