Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain

• URL: http://arxiv.org/abs/2407.11941v1
• Date: Tue, 16 Jul 2024 17:29:24 GMT
• Title: Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain
• Authors: Marco Huber, Naser Damer,
• Abstract summary: We take a step forward and investigate explainable face recognition in the unexplored frequency domain. This makes this work the first to propose explainability of verification-based decisions in the frequency domain.
• Score: 6.69421628320396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The need for more transparent face recognition (FR), along with other visual-based decision-making systems has recently attracted more attention in research, society, and industry. The reasons why two face images are matched or not matched by a deep learning-based face recognition system are not obvious due to the high number of parameters and the complexity of the models. However, it is important for users, operators, and developers to ensure trust and accountability of the system and to analyze drawbacks such as biased behavior. While many previous works use spatial semantic maps to highlight the regions that have a significant influence on the decision of the face recognition system, frequency components which are also considered by CNNs, are neglected. In this work, we take a step forward and investigate explainable face recognition in the unexplored frequency domain. This makes this work the first to propose explainability of verification-based decisions in the frequency domain, thus explaining the relative influence of the frequency components of each input toward the obtained outcome. To achieve this, we manipulate face images in the spatial frequency domain and investigate the impact on verification outcomes. In extensive quantitative experiments, along with investigating two special scenarios cases, cross-resolution FR and morphing attacks (the latter in supplementary material), we observe the applicability of our proposed frequency-based explanations.

Related papers

• Towards A Comprehensive Visual Saliency Explanation Framework for AI-based Face Recognition Systems [9.105950041800225]
  This manuscript conceives a comprehensive explanation framework for face recognition tasks. An exhaustive definition of visual saliency map-based explanations for AI-based face recognition systems is provided. A new model-agnostic explanation method named CorrRISE is proposed to produce saliency maps.
  arXiv  Detail & Related papers  (2024-07-08T14:25:46Z)
• Explainable Face Verification via Feature-Guided Gradient Backpropagation [9.105950041800225]
  There is a growing need for reliable interpretations of decisions of face recognition systems. This paper first explores the spatial relationship between face image and its deep representation via gradient backpropagation. A new explanation approach has been conceived, which provides precise and insightful similarity and dissimilarity saliency maps to explain the "Accept" and "Reject" decision of an FR system.
  arXiv  Detail & Related papers  (2024-03-07T14:43:40Z)
• Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection [96.539862328788]
  Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF) Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-21T08:58:49Z)
• Explanation of Face Recognition via Saliency Maps [13.334500258498798]
  This paper proposes a rigorous definition of explainable face recognition (XFR) It then introduces a similarity-based RISE algorithm (S-RISE) to produce high-quality visual saliency maps. An evaluation approach is proposed to systematically validate the reliability and accuracy of general visual saliency-based XFR methods.
  arXiv  Detail & Related papers  (2023-04-12T19:04:21Z)
• Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain [77.8858706250075]
  This paper proposes a privacy-preserving face recognition method using differential privacy in the frequency domain. Our method performs very well with several classical face recognition test sets.
  arXiv  Detail & Related papers  (2022-07-15T07:15:36Z)
• Masked Frequency Modeling for Self-Supervised Visual Pre-Training [102.89756957704138]
  We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models. MFM first masks out a portion of frequency components of the input image and then predicts the missing frequencies on the frequency spectrum. For the first time, MFM demonstrates that, for both ViT and CNN, a simple non-Siamese framework can learn meaningful representations even using none of the following: (i) extra data, (ii) extra model, (iii) mask token.
  arXiv  Detail & Related papers  (2022-06-15T17:58:30Z)
• Beyond the Visible: A Survey on Cross-spectral Face Recognition [15.469814029453893]
  Cross-spectral face recognition (CFR) refers to recognizing individuals using face images stemming from different spectral bands. Recent advances in deep neural networks (DNNs) have resulted in significant improvement in the performance of CFR systems.
  arXiv  Detail & Related papers  (2022-01-12T12:09:24Z)
• Detect and Locate: A Face Anti-Manipulation Approach with Semantic and Noise-level Supervision [67.73180660609844]
  We propose a conceptually simple but effective method to efficiently detect forged faces in an image. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
  arXiv  Detail & Related papers  (2021-07-13T02:59:31Z)
• Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain [88.7339322596758]
  We present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery. SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
  arXiv  Detail & Related papers  (2021-03-02T16:45:08Z)
• Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues [44.78809141563589]
  We propose a novel Frequency in Face Forgery Network (F3-Net), taking advantages of two different but complementary frequency-aware clues. We show that the proposed F3-Net significantly outperforms competing state-of-the-art methods on all compression qualities in the challenging FaceForensics++ dataset.
  arXiv  Detail & Related papers  (2020-07-18T07:39:08Z)
• Learning Oracle Attention for High-fidelity Face Completion [121.72704525675047]
  We design a comprehensive framework for face completion based on the U-Net structure. We propose a dual spatial attention module to efficiently learn the correlations between facial textures at multiple scales. We take the location of the facial components as prior knowledge and impose a multi-discriminator on these regions.
  arXiv  Detail & Related papers  (2020-03-31T01:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.