FACE-AUDITOR: Data Auditing in Facial Recognition Systems

• URL: http://arxiv.org/abs/2304.02782v1
• Date: Wed, 5 Apr 2023 23:03:54 GMT
• Title: FACE-AUDITOR: Data Auditing in Facial Recognition Systems
• Authors: Min Chen and Zhikun Zhang and Tianhao Wang and Michael Backes and Yang Zhang
• Abstract summary: Few-shot-based facial recognition systems have gained increasing attention due to their scalability and ability to work with a few face images. To prevent the face images from being misused, one straightforward approach is to modify the raw face images before sharing them. We propose a complete toolkit FACE-AUDITOR that can query the few-shot-based facial recognition model and determine whether any of a user's face images is used in training the model.
• Score: 24.082527732931677
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Few-shot-based facial recognition systems have gained increasing attention due to their scalability and ability to work with a few face images during the model deployment phase. However, the power of facial recognition systems enables entities with moderate resources to canvas the Internet and build well-performed facial recognition models without people's awareness and consent. To prevent the face images from being misused, one straightforward approach is to modify the raw face images before sharing them, which inevitably destroys the semantic information, increases the difficulty of retroactivity, and is still prone to adaptive attacks. Therefore, an auditing method that does not interfere with the facial recognition model's utility and cannot be quickly bypassed is urgently needed. In this paper, we formulate the auditing process as a user-level membership inference problem and propose a complete toolkit FACE-AUDITOR that can carefully choose the probing set to query the few-shot-based facial recognition model and determine whether any of a user's face images is used in training the model. We further propose to use the similarity scores between the original face images as reference information to improve the auditing performance. Extensive experiments on multiple real-world face image datasets show that FACE-AUDITOR can achieve auditing accuracy of up to $99\%$. Finally, we show that FACE-AUDITOR is robust in the presence of several perturbation mechanisms to the training images or the target models. The source code of our experiments can be found at \url{https://github.com/MinChen00/Face-Auditor}.

Related papers

• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration. We propose OSDFace, a novel one-step diffusion model for face restoration. Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• CLIP Unreasonable Potential in Single-Shot Face Recognition [0.0]
  Face recognition is a core task in computer vision designed to identify and authenticate individuals by analyzing facial patterns and features. Recent Contrastive Language Image Pretraining (CLIP) a model developed by OpenAI has shown promising advancements. CLIP links natural language processing with vision tasks allowing it to generalize across modalities.
  arXiv  Detail & Related papers  (2024-11-19T08:23:52Z)
• ID-Guard: A Universal Framework for Combating Facial Manipulation via Breaking Identification [60.73617868629575]
  misuse of deep learning-based facial manipulation poses a potential threat to civil rights. To prevent this fraud at its source, proactive defense technology was proposed to disrupt the manipulation process. We propose a novel universal framework for combating facial manipulation, called ID-Guard.
  arXiv  Detail & Related papers  (2024-09-20T09:30:08Z)
• Exploring Decision-based Black-box Attacks on Face Forgery Detection [53.181920529225906]
  Face forgery generation technologies generate vivid faces, which have raised public concerns about security and privacy. Although face forgery detection has successfully distinguished fake faces, recent studies have demonstrated that face forgery detectors are very vulnerable to adversarial examples.
  arXiv  Detail & Related papers  (2023-10-18T14:49:54Z)
• A Comparative Analysis of the Face Recognition Methods in Video Surveillance Scenarios [0.0]
  This study presents comparative benchmark tables for the state-of-art face recognition methods. We constructed a video surveillance dataset of face IDs with high age variance, intra-class variance (face make-up, beard, etc.) with native surveillance facial imagery data for evaluation. On the other hand, this work discovers the best recognition methods for different conditions like non-masked faces, masked faces, and faces with glasses.
  arXiv  Detail & Related papers  (2022-11-05T17:59:18Z)
• DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain [23.4606547767188]
  DuetFace is a privacy-preserving face recognition method that employs collaborative inference in the frequency domain. The proposed method achieves a comparable recognition accuracy and cost to the unprotected ArcFace and outperforms the state-of-the-art privacy-preserving methods.
  arXiv  Detail & Related papers  (2022-07-15T08:35:44Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)
• Harnessing Unrecognizable Faces for Face Recognition [87.80037162457427]
  We propose a measure of recognizability of a face image, implemented by a deep neural network trained using mostly recognizable identities. We show that accounting for recognizability reduces error rate of single-image face recognition by 58% at FAR=1e-5.
  arXiv  Detail & Related papers  (2021-06-08T05:25:03Z)
• Real Time Face Recognition Using Convoluted Neural Networks [0.0]
  Convolutional Neural Networks are proved to be best for facial recognition. The creation of dataset is done by converting face videos of the persons to be recognized into hundreds of images of person.
  arXiv  Detail & Related papers  (2020-10-09T12:04:49Z)
• Towards Face Encryption by Generating Adversarial Identity Masks [53.82211571716117]
  We propose a targeted identity-protection iterative method (TIP-IM) to generate adversarial identity masks. TIP-IM provides 95%+ protection success rate against various state-of-the-art face recognition models.
  arXiv  Detail & Related papers  (2020-03-15T12:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.