Inverting Black-Box Face Recognition Systems via Zero-Order Optimization in Eigenface Space

• URL: http://arxiv.org/abs/2506.09777v1
• Date: Wed, 11 Jun 2025 14:15:18 GMT
• Title: Inverting Black-Box Face Recognition Systems via Zero-Order Optimization in Eigenface Space
• Authors: Anton Razzhigaev, Matvey Mikhalchuk, Klim Kireev, Igor Udovichenko, Andrey Kuznetsov, Aleksandr Petiushko,
• Abstract summary: Reconstructing facial images from black-box recognition models poses a significant privacy threat.<n>This paper introduces DarkerBB, a novel approach that reconstructs color faces by performing zero-order optimization within a PCA-derived eigenface space.<n>Experiments on LFW, AgeDB-30, andFP benchmarks demonstrate that DarkerBB achieves state-of-the-art verification accuracies in the similarity-only setting, with competitive query efficiency.
• Score: 43.698488201196746
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconstructing facial images from black-box recognition models poses a significant privacy threat. While many methods require access to embeddings, we address the more challenging scenario of model inversion using only similarity scores. This paper introduces DarkerBB, a novel approach that reconstructs color faces by performing zero-order optimization within a PCA-derived eigenface space. Despite this highly limited information, experiments on LFW, AgeDB-30, and CFP-FP benchmarks demonstrate that DarkerBB achieves state-of-the-art verification accuracies in the similarity-only setting, with competitive query efficiency.

Related papers

• Privacy-preserving Preselection for Face Identification Based on Packing [9.235015111013064]
  We propose a novel and efficient scheme for face retrieval in the ciphertext domain, termed Privacy-Preserving Preselection for Face Identification Based on Packing (PFIP)<n>PFIP incorporates an innovative preselection mechanism to reduce computational overhead and a packing module to enhance the flexibility of biometric systems during the enrollment stage.<n>Experiments conducted on the LFW and CASIA datasets demonstrate that PFIP preserves the accuracy of the original face recognition model, achieving a 100% hit rate while retrieving 1,000 ciphertext face templates within 300 milliseconds.
  arXiv  Detail & Related papers  (2025-07-03T08:15:07Z)
• xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devices [4.910937238451485]
  Heterogeneous Face Recognition (HFR) addresses the challenge of matching face images across different sensing modalities.<n>We present a lightweight yet effective HFR framework by adapting a hybrid CNN-Transformer architecture.<n>Our approach enables efficient end-to-end training with minimal paired heterogeneous data while preserving strong performance on standard RGB face recognition tasks.
  arXiv  Detail & Related papers  (2025-04-28T10:03:11Z)
• DiffUMI: Training-Free Universal Model Inversion via Unconditional Diffusion for Face Recognition [17.70133779192382]
  We introduce DiffUMI, a diffusion-based universal model inversion attack that requires no additional training.<n>It surpasses state-of-the-art attacks by 15.5% and 9.82% in success rate on standard and privacy-preserving face recognition systems, respectively.
  arXiv  Detail & Related papers  (2025-04-25T01:53:27Z)
• ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Models [14.137615267026755]
  We propose Zeroth-order Intrinsic-dimensional Prompt-tuning, which enables efficient and robust prompt optimization in a purely black-box setting.<n>We evaluate ZIP on 13+ vision-language tasks in standard benchmarks and show that it achieves an average improvement of approximately 6% in few-shot accuracy and 48% in query efficiency.
  arXiv  Detail & Related papers  (2025-04-09T12:56:22Z)
• Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis [48.59382455101753]
  2D face recognition encounters challenges in unconstrained environments due to varying illumination, occlusion, and pose. Recent studies focus on RGB-D face recognition to improve robustness by incorporating depth information. In this work, we first construct a diverse depth dataset generated by 3D Morphable Models for depth model pre-training. Then, we propose a domain-independent pre-training framework that utilizes readily available pre-trained RGB and depth models to separately perform face recognition without needing additional paired data for retraining.
  arXiv  Detail & Related papers  (2024-03-11T09:12:24Z)
• Enhancing Generalization of Invisible Facial Privacy Cloak via Gradient Accumulation [46.81652932809355]
  A new type of adversarial privacy cloak (class-universal) can be applied to all the images of regular users. We propose Gradient Accumulation (GA) to aggregate multiple small-batch gradients into a one-step iterative gradient to enhance the gradient stability and reduce the usage of quantization operations. Experiments show that our proposed method achieves high performance on the Privacy-Commons dataset against black-box face recognition models.
  arXiv  Detail & Related papers  (2024-01-03T07:00:32Z)
• PRO-Face S: Privacy-preserving Reversible Obfuscation of Face Images via Secure Flow [69.78820726573935]
  We name it PRO-Face S, short for Privacy-preserving Reversible Obfuscation of Face images via Secure flow-based model. In the framework, an Invertible Neural Network (INN) is utilized to process the input image along with its pre-obfuscated form, and generate the privacy protected image that visually approximates to the pre-obfuscated one.
  arXiv  Detail & Related papers  (2023-07-18T10:55:54Z)
• Fine-Grained Visual Prompting [35.032567257651515]
  Fine-Grained Visual Prompting (FGVP) demonstrates superior performance in zero-shot comprehension of referring expressions. It outperforms prior methods by an average margin of 3.0% to 4.6%, with a maximum improvement of 12.5% on the RefCOCO+ testA subset.
  arXiv  Detail & Related papers  (2023-06-07T11:39:56Z)
• Blind Face Restoration: Benchmark Datasets and a Baseline Model [63.053331687284064]
  Blind Face Restoration (BFR) aims to construct a high-quality (HQ) face image from its corresponding low-quality (LQ) input. We first synthesize two blind face restoration benchmark datasets called EDFace-Celeb-1M (BFR128) and EDFace-Celeb-150K (BFR512) State-of-the-art methods are benchmarked on them under five settings including blur, noise, low resolution, JPEG compression artifacts, and the combination of them (full degradation)
  arXiv  Detail & Related papers  (2022-06-08T06:34:24Z)
• Darker than Black-Box: Face Reconstruction from Similarity Queries [65.62256987706128]
  We propose a novel approach that allows reconstructing the face querying only similarity scores of the black-box model. While our algorithm operates in a more general setup, experiments show that it is query efficient and outperforms the existing methods.
  arXiv  Detail & Related papers  (2021-06-27T17:25:46Z)
• Black-Box Face Recovery from Identity Features [61.950765357647605]
  We attack the state-of-the-art face recognition system (ArcFace) to test our algorithm. Our algorithm requires a significantly less number of queries compared to the state-of-the-art solution.
  arXiv  Detail & Related papers  (2020-07-27T15:25:38Z)
• Creating Artificial Modalities to Solve RGB Liveness [79.9255035557979]
  We introduce two types of artificial transforms: rank pooling and optical flow, combined in end-to-end pipeline for spoof detection. The proposed method achieves state-of-the-art on the largest cross-ethnicity face anti-spoofing dataset CASIA-SURF CeFA (RGB)
  arXiv  Detail & Related papers  (2020-06-29T13:19:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.